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SUMMARY 


Problem 


An  Important  element  of  personnel  policy  testing  and  decision  making 
with  computer  simulation  models  is  the  criteria  for  evaluating  changes. 

The  ultimate  criterion  is  the  "goodness  of  fit"  of  the  enlisted  personnel 
inventory  to  established  personnel  requirements  within  other  external 
constraints  such  as  end  strength,  man-year  average,  and  control  dollars. 
Within  these  constraints,  however,  there  is  a considerable  latitude  for 
discretionary  action  for  the  decision  maker,  especially  on  decisions  af- 
fecting the  length  of  service  (LOS)  dimension  of  the  force. 

Measures  of  cost  and  inventory  have  been  used  by  the  Bureau  of  Naval 
Personnel  (BUPERS)  to  manage  the  distribution  of  personnel  filling  the 
billet  structure.  However,  measures  of  cost  and  inventory  by  themselves 
are  not  sufficient  for  personnel  planning.  An  optimum  distribution  of 
strength  cannot  be  achieved  by  simply  minimizing  cost  while  maximizing 
inventory.  Other  measures  must  be  identified,  such  as  a measure  of  utility. 

Obj ective 

The  objective  was  to  determine  the  productivity  of  an  average  enlisted 
man  in  terms  of  utility  to  the  Navy  as  he  progressed  in  years  of  service 
in  a specific  paygrade.  The  results  are  to  be  incorporated  in  a force 
distribution  optimization  model  to  be  used  by  the  Bureau  of  Naval  Personnel. 

Approach 

An  Important  prerequisite  for  developing  such  a model  is  a measure  which 
can  be  used  to  quantify,  over  a useful  range,  the  value  of  the  average  Navy 
enlisted  man.  Lacking  an  available  source  for  a measure  of  this  kind,  it 
was  determined  that  the  only  practical  source  from  which  a measure  might  be 
constructed  were  Navy  experts — those  who  have  had  an  opportunity  to  monitor 
the  progress  of  the  enlisted  man  and  to  observe  the  change  in  his  usefulness 
to  the  Navy  throughout  his  career.  These  are  the  only  experts  whose  opinions 
and  observations  are  likely  to  be  valid  and  credible. 

Two  Delphi  experiments  and  a Broadcast  experiment  were  conducted  which 
solicited  opinions  of  Navy  experts  from  headquarters  as  well  as  the  Fleet. 
These  experts  were  asked  to  estimate  (1)  the  value  a man  has  at  entry  into 
the  paygrade,  (2)  the  year  of  service  where  he  reaches  maximum  value  in 
his  paygrade,  (3)  the  length  of  service  during  which  he  maintains  his 
maximum  value,  and  (4)  the  value  he  has  at  the  end  of  his  career,  assuming 
he  remains  in  the  same  paygrade.  Utility  estimates  of  the  average  man  in 
each  paygrade  relative  to  the  utility  of  the  average  man  in  the  highest 
paygrade  (E-9)  were  also  obtained. 
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Two  panels  of  experts,  a senior  officer  and  a junior  officer  panel, 
were  selected  within  BUPERS  for  the  Delphi  I experiment.  The  Broadcast 
experiment  solicited  utility  estimates  from  a different  population  of 
Navy  experts  (including  senior  enlisted  men)  to  validate  and  refine  the 
results  of  the  Delphi  I experiment.  A one-shot  questionnaire  was  used 
this  time  rather  than  the  iterative  Delphi  procedure.  Length  of  service 
was  divided  into  four  classes:  (1)  first  term,  (2)  second  term,  (3) 

careerist  with  less  than  20  years,  and  (4)  careerist  with  20  to  30  years. 
Estimates  were  obtained  for  annual  base  pay,  bonus  pay,  and  achievement. 
The  Delphi  II  experiment  was  addressed  only  to  three  panels  of  personnel 
serving  in  operational  billets  in  the  Fleet.  Seventy-five  officers  and 
enlisted  men  serving  in  the  Second  Fleet  participated  in  the  third 
experiment. 

Results 


a.  In  describing  the  utility  of  the  average  enlisted  man  as  he  advanced 
in  years  of  service,  little  statistical  difference  was  found  among  skill 
groupings  (Delphi  I experiment). 

b.  In  the  Broadcast  experiment,  one  proxy  for  utility,  base  pay,  was 
found  to  be  biased.  Utility  curves  constructed  from  the  base  pay  proxy 
were  found  to  be  linearly  related  to  an  analogous  table  of  values  con- 
structed from  actual  base  pay. 

c.  Two  other  proxies  for  utility  used  in  the  Broadcast  experiment, 
bonus  pay  and  achievement,  both  produced  utility  curves  similar  to  that 
of  the  utility  curves  developed  in  the  Delphi  experiments. 

d.  Results  of  both  Delphi  experiments  showed  that  the  average  enlisted 
progressor  reaches  a point  of  maximum  utility  at  approximately  24-27  years 
of  service  and  thereafter  declines  in  utility.  This  general  pattern  also 
applies  to  the  average  enlisted  man  at  each  paygrade.  Those  who  do  not 
get  promoted  (i.e.,  those  who  remain  in  grade)  decline  in  value  to  the 
Navy  at  advanced  years  of  service. 

Conclusions 


From  the  consistency  of  the  results  in  both  Delphi  experiments,  it  can 
be  concluded  that  the  Delphi  technique  seems  to  be  a valid  technique  to 
measure  the  utility  of  the  enlisted  man  by  paygrade  and  LOS. 

Since  an  enlisted  man's  usefulness  decreases  if  he  remains  in  the  same 
paygrade  (i.e.,  does  not  get  promoted)  after  a period  of  time,  the  Navy 
could  possibly  benefit  by  employing  "early  out"  policies  for  enlisted  men 
with  no  chance  of  promotion. 
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Recommendations 


The  results  of  this  study  should  be  used  to  develop  a utility  model 
to  help  the  Bureau  of  Naval  Personnel  define  long-range  management  goals 
for  the  Navy  Personnel  enlisted  force.  The  utility  model  could  be  used 
along  with  other  models  such  a?  cost,  inventory,  and  reenlistment 
elasticity  models  to  optimize  personnel  force  distributions  by  skill 
grouping,  length  of  service,  and  paygrade. 
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INTRODUCTION 


Problem 


An  important  element  of  personnel  policy  testing  and  decision  making 
with  computer  simulation  models  is  the  criteria  used  for  evaluating 
changes.  One  criterion,  cost,  is  provided  through  the  use  of  the  Per 
Capita  Cost  Model  (Dept,  of  Navy,  1972).  The  ultimate  criterion  is  the 
"goodness  of  fit"  of  the  enlisted  personnel  inventory  to  established 
personnel  requirements,  within  other  external  constraints  such  as  end 
strength,  man-year  average,  and  control  dollars.  Within  these  con- 
straints, however,  there  is  considerable  latitude  for  discretionary 
action  for  the  decision  maker,  especially  on  decisions  affecting  the 
length  of  service  (LOS)  dimension  of  the  force. 

An  Enlisted  Personnel  Planning  System  (ADSTAP)  was  developed 
(Silverman,  1971)  to  manage  the  distribution  of  personnel  filling  the 
billet  structure.  ADSTAP  encompasses  a series  of  models  for  projecting 
the  personnel  inventory  distribution  of  the  enlisted  force,  and  the  Per 
Capita  Cost  Model  for  computing  the  associated  costs.  Each  of  these 
models  can  be  exercised  against  various  recruitment,  reenlistment,  and 
retirement  options,  thus  enabling  an  assessment  of  alternative  policies 
with  regard  to  their  impact  on  the  inventory  distribution  and  costs  of 
the  enlisted  force. 

However,  measures  of  cost  and  inventory,  by  themselves,  are  not  suf- 
ficient for  personnel  planning.  An  optimum  distribution  of  strength 
cannot  be  achieved  by  simply  minimizing  cost  while  maximizing  inventory. 

An  additional  measure  is  needed  to  determine  the  value  of  a man' working 
a job.  In  industry,  the  value  may  be  determined  by  the  prevailing  wage 
for  that  job.  In  the  military,  however,  no  direct  control  is  exercised 
over  what  is  paid  to  the  Navy  enlisted  man.  Therefore,  what  the  military 
pays  for  labor  is  not  necessarily  a measure  of  the  value  that  is  attributed 
to  that  labor.  Certainly,  the  pay  scale  is  not  irrational — the  Navy  must 
believe  an  E-7  is  worth  more  than  an  E-l  because  of  the  higher  pay  an  E-7 
receives.  The  question  remains,  "How  much  more  is  he  worth?" 

Obj  ectlve 

The  objective  was  to  determine  the  productivity  of  an  average  enlisted 
man  in  terms  of  utility  to  the  Navy  as  he  progresses  in  a specific  pay- 
grade  in  years  of  service.  The  utility  results  are  to  be  used  in  an  over- 
all force  distribution  optimization  model.  This  model  would  enable  the 
personnel  planner  to  assess  the  overall  value  that  the  Navy  should  expect 
to  receive  from  the  personnel  inventory  that  the  manpower  dollars  expended 
afford. 


APPROACH 


An  important  prerequisite  for  developing  a utility  model  is  a measure 
which  can  be  used  to  quantify,  over  a useful  range,  the  value  of  the 
average  Navy  enlisted  man.  It  was  determined  that  the  opinions  of  Navy 
experts  were  the  only  practical  source  from  which  such  a measure  might 
be  constructed.  Since  these  men  have  had  an  opportunity  to  monitor  the 
progress  of  the  enlisted  man  and  to  observe  the  change  in  his  usefulness 
to  the  Navy  throughout  his  career,  their  opinions  and  observations  are 
likely  to  be  valid  and  credible.  Thus,  two  Delphi  experiments  and  a 
Broadcast  experiment  were  performed  to  solicit  the  opinions  of  these 
experts  in  a procedural  attempt  to  develop  the  value  measure.  The  Delphi 
methodology  developed  at  Rand  Corporation  was  generally  followed.  In 
all  experiments,  the  experts  were  asked  to  estimate  (1)  the  value  a man 
has  at  entry  into  a paygrade,  (2)  the  year  of  service  when  he  reaches 
maximum  value  in  a paygrade,  (3)  the  length  of  service  during  which  he 
maintains  his  maximum  value,  and  (A)  the  value  he  has  at  the  end  of  his 
career,  assuming  he  remains  in  the  same  paygrade. 
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DISCUSSION  AND  RESULTS 


Delphi  I Experiment 


In  the  Delphi  I experiment  (B-K  Dynamics,  1973),  two  panels  of 
experts — one  comprised  of  senior  officers,  and  the  other,  junior 
officers — were  selected  within  BUPERS.  All  of  these  officers  had 
completed  operational  fleet  assignments  within  the  last  year  and  a half. 
The  senior  officer  panel  consisted  of  former  COs  and  XOs,  and  the  junior 
officer  panel,  of  officers  who  had  previously  been  division  officers  and 
department  heads.  Throughout  the  Delphi  I experiment,  the  responses  of 
these  two  panels  were  kept  separate. 

The  questionnaire  developed  for  administration  to  the  selected 
panelists  explained  the  utility  concept  and  solicited  each  participant's 
estimates  of  personnel  utility  in  two  formats — graphic  and  tabular. 
Further,  each  participant  was  asked  to  make  two  sets  of  estimates:  one 

considering  the  "hard"  skill  or  technical  enlisted  man  and  the  other  the 
"soft"  skill  or  nontechnical  enlisted  man. 

To  complete  the  graphical  format,  each  participant  was  asked  to 
describe  che  utility  of  the  average  Navy  enlisted  man  as  he  advanced  in 
years  of  service.  The  participant  was  directed  to  sketch  this  observed 
change  in  value  on  a scale  ranging  from  0 to  100  over  the  years  of  service 
of  an  average  Navy  enlisted  man.  The  restrictions  placed  on  his  sketch 
were:  (1)  the  curve  should  begin  at  0,  0 (interpreted  as  0 value  at  0 

years  in  service),  and  (2)  at  some  point  in  time  from  the  years  1 through 
30,  it  should  reach  a maximum  excursion  of  100,  which  would  indicate  the 
point  in  the  career  of  the  average  Navy  enlisted  man  when  he  reaches 
maximum  value  to  the  Navy. 

To  complete  the  tabular  format,  the  participant  was  directed  to  supply 
estimates  of  usefulness  for  each  paygrade.  In  other  words,  he  was  to 
estimate  the  value  of  the  average  Navy  enlisted  man  in  each  grade  (E-l 
through  E-9),  regardless  of  length  of  service  in  that  grade,  relative  to 
all  the  grades,  on  a scale  from  0 to  100.  Further,  participants  were 
asked  to  identify  any  materials  or  information  which  could  assist  them  in 
their  assessment.  This  information  was  to  be  supplied,  if  possible,  by 
those  administering  the  questionnaire.  Upon  completion  and  return  of 
the  first-round  questionnaires,  the  individual  estimates  were  aggregated 
by  panel  and  by  skill.  The  mean  utility  estimates  and  the  interquartile 
statistics  were  computed,  plotted,  and  tabulated. 

Second-round  questionnaires  for  each  panel  were  then  prepared,  incor- 
porating these  statistics.  In  this  round,  the  participants  were  asked  to 
reappraise  their  estimates,  based  on  the  aggregate  statistics  for  the 
panel  in  which  they  served,  and  to  simply  resketch  their  own  estimates  of 
personnel  utility  over  the  aggregate  graphical  presentation.  They  were 
also  asked  to  supply  the  rationale  they  had  employed  in  making  their 
utility  assessments. 
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Upon  return  of  the  second-round  questionnaires,  the  aggregate 
statistics  again  were  prepared.  Further,  an  edited  summary  of  rationale 
provided  by  the  panel  members  was  prepared.  These  statistics  and  sum- 
mary rationale  were  incorporated  into  the  third-round  questionnaires 
and  distributed  to  the  panelists.  The  participants  were  directed  to 
again  reappraise  their  estimates  and  to  respond  to  the  summary  rationale 
indicating  their  agreement  or  disagreement. 

Analysis  of  the  third-round  statistics  disclosed  considerable  con- 
vergence in  the  panel  estimates,  as  illustrated  in  the  narrowness  of  the 
interquartile  statistics.  Because  of  this  convergence,  and  the  fact 
that  a number  of  the  panel  members  had  left  BUPEKS,  it  was  determined 
that  no  greater  refinement  of  the  utility  estimates  from  this  first 
experiment  was  necessary. 

Two  significant  results  were  determined  in  analysis  of  the  returns. 
First,  a statistical  difference  was  found  between  the  estimates  aggregated 
by  panel.  Junior  officers  placed  a greater  emphasis  on  experience  than 
senior  officers.  Second,  little  statistical  difference  was  found  between 
the  panel  estimates  aggregated  by  skill  grouping.  As  a consequence,  for 
further  use,  the  estimates  were  aggregated  by  dropping  the  differentiation 
by  skill.  The  means  and  interquartiles  of  the  resultant  utility  curves 
are  shown  in  Figures  1 and  2. 

Broadcast  Experiment 

The  purpose  of  the  Broadcast  experiment  was  to  validate  and  refine, 
if  possible,  the  results  of  the  Delphi  I experiment.  The  methodology 
employed  was  to  solicit  utility  estimates  in  a different  manner  and  from 
a different  population  of  Navy  Experts,  and  to  analyze  these  resultant 
estimates  for  consistency  and  comparability  both  within  the  experiment 
and  between  experiments. 

The  first  major  difference  between  the  two  experiments  was  the  means 
used  to  select  utility  estimates.  In  contrast  to  the  iterative  Delphi 
procedure  used  in  the  first  experiment,  a one-shot  questionnaire  was 
developed  and  administered  to  several  distinct  groups  of  experts.  It 
was  recognized  that  this  procedure  would  not  yield  aggregate  results  with 
the  same  narrow  dispersion  observed  in  the  iterative  procedure.  However, 
it  was  believed  that  the  mean  value  estimates  of  the  larger  sample  of 
experts  available  with  the  simpler  one-shot  procedure  would  be  sufficient 
for  testing  consistency  and  comparability.  Both  pilot  and  validation  tests 
were  administered. 
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The  second  major  difference  was  the  unit  of  measure  in  which  utility 
estimates  were  solicited.  In  the  first  experiment,  participants  used  a 
nondimensional  scale  ranging  from  0 to  100.  In  the  second  experiment, 
three  proxies  for  utility — base  pay,  bonus  pay,  and  achievement — were 
used.^  It  was  thought  that  participants  would  find  it  easier  to  deal 
with  such  measures  in  a one-shot  exercise. 

The  third  major  difference  was  that  estimates  were  obtained  from  a 
wider  population  of  Navy  experts,  including  senior  enlisted  men.  To 
accomplish  this,  the  questionnaire  was  administered  to  four  Navy  panels, 
comprised  of  personnel  from  the  following  activities: 

1.  Training  Publications  Group  (E-7's,  E-8's,  E-9's) 

2.  Examination  Center  (E-9's) 

3.  Naval  Postgraduate  School  (Student  Officers) — Management  Science 

4.  Naval  Postgraduate  School  (Student  Officers) — Operations  Research 

The  first  proxy  measure  included  in  the  questionnaire  was  base  pay. 
Each  participant  was  directed  to  construct  a base  pay  table,  recording 
dollar  values  which  corresponded  to  his  judgment  of  what  the  average 
enlisted  man  should  be  paid.  The  pay  table  was  to  reflect  what  the 
participant  perceives  to  be  the  value  of  the  average  enlisted  man. 

The  matrix  illustrated  in  Figure  3 was  used  by  the  participants  to 
record  their  estimates.  In  the  first  experiment,  it  was  observed  that 
participants  experienced  some  difficulty  in  estimating  the  utility  of  an 
enlisted  man  by  length  of  service.  This  difficulty  apparently  was  due  to 
the  fact  that  there  is  no  readily  available  measures  of  the  value  of 
length  of  service  whereas  the  grade  of  an  enlisted  man  is  an  apparent 
measure.  Because  of  this  built-in  difficulty,  length  of  service  as  a 
factor  for  use  in  the  second  experiment  was  subdivided  into  the  following 
four  classes: 

1.  First  Term 

2.  Second  Term 

3.  Careerist  with  less  than  20  years 

4.  Careerist  with  20  to  30  years 


^Only  base  pay  and  achievement  were  included  in  the  pilot  test.  Bonus 
pay  was  added  to  the  validation  test. 
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Figure  3.  Matrix  used  for  estimating  what  the  average  enlisted  man 
should  be  paid  in  comparison  to  the  careerist  journeyman. 


By  the  same  rationale,  the  pay  grade  of  the  average  enlisted  man  was 
aggregated  into  three  categories.  Rather  than  estimating  the  utility  of 
an  average  enlisted  man  differentiated  by  paygrade,  each  participant  was 
requested  to  estimate  the  utility  of  an  enlisted  man  differentiated  by 
the  type  of  job  he  is  performing — i.e.,  whether  he  is  an  apprentice,  a 
journeyman,  or  a supervisor.  For  this  purpose,  the  journeyman  was  defined 
to  be  the  enlisted  man  who  actually  performs  the  "hands-on-labor."  In 
aggregating  the  results  of  this  experiment  using  existing  paygrade  popula- 
tion statistics,  the  following  equivalencies  were  used:  apprentices — E-3's 

and  E-4's;  journeymen — E-5's  and  E-6’s;  and  supervisors — E-7's,  E-8's  and 
E-9's. 

The  base  pay  of  a journeyman  with  less  than  20  years  experience  was  set 
arbitrarily  at  $10,000  per  year.  The  participants  were  asked  to  accept 
this  figure  and  to  record  their  base  pay  estimates  for  each  of  the  other 
classifications  of  rate  and  experience  relative  to  this  preset  figure. 

Each  participant  was  asked  to  construct  two  base  pay  tables.  The  first 
was  to  be  constructed  considering  the  average  enlisted  man  in  the  overall 
Navy,  and  the  second,  considering  the  average  enlisted  man  in  one  of  five 
rating  groups.  Participants  were  to  select  a rating  group,  based  on  their 
own  experience,  from  one  of  the  following: 

1.  Electronic  Equipment  Repair 

2.  Electrical/Mechanical  Equipment  Repair 

3.  Seamanship  Specialists 

4.  Administrative  Specialists  and  Clerks 

5.  Service  and  Supply  Handlers 
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Tht*  results  from  both  the  pilot  and  validation  tests  of  the  question- 
naire showed  that  the  base  pay  tables  constructed  by  the  participants 
were  linearly  related  to  the  analogous  table  of  values  constructed  from 
the  actual  base  pay.  The  participants'  tables  also  reflected  actual  pay 
differentials  between  rates  and  years  of  service,  thus  accounting  for  the 
constant  of  proportionality. 

To  analyze  the  data  obtained  from  the  base  pay  question,  figures  from 
the  tables  submitted  by  participants  were  converted  to  a scale  ranging 
from  0 to  100.  The  individual  tables  were  then  aggregated  and  averaged 
to  construct  summary  tables  of  values  by  panel  and  by  rating  group.  All 
of  the  tables  developed  from  the  base  pay  estimates,  whether  aggregated 
by  panel,  rating  group,  or  both,  were  strikingly  similar. 

When  the  base  pay  utility  tables  were  transformed  into  LOS-dependent 
estimates  (by  computing  a weighted  average  utility  at  each  year  of  service 
according  to  the  actual  strength  distribution  at  each  year  of  service), 
the  resultant  curve  showed  nearly  linear  growth  in  utility  from  the  point 
of  entry  into  the  Navy  up  to  the  30-year  point.  Because  of  this  finding, 
there  was  some  concern  as  to  whether  base  pay  was  a biased  proxy  for 
utility.  There  are  a number  of  explanations  for  this  phenomenon.  First, 
since  participants  know  generally  what  enlisted  men  are  actually  paid  as 
they  advance  in  rate  and  experience,  this  idea  might  have  overwhelmed  any 
different  notion  that  they  may  have  had  regarding  what  enlisted  men  should 
be  paid.  On  the  other  hand,  participants  may  believe  that  the  military  is 
receiving  full  value  from  each  manpower  dollar  expended,  and  that  the 
actual  base  pay  of  the  average  enlisted  man  is,  in  fact,  proportional  to  the 
utility  derived. 

In  an  effort  to  avoid  this  apparent  bias,  a "bonus  pay"  question  was 
included  in  the  validation  experiment.  To  complete  this  question,  the 
participant  was  directed  to  construct  a bonus  pay  table,  using  the  same 
format  as  that  used  for  the  base  pay  table,  with  the  exception  that  no 
values  were  pre-recorded.  The  participant  was  directed  to  decide  when  the 
average  enlisted  man  reaches  his  greatest  value  to  the  Navy  and  to  record 
in  the  table  a value  of  $100  for  the  appropriate  experience-rate  groupings. 
The  rest  of  the  table  was  to  be  completed  by  recording  the  bonus  pay  that 
the  average  Navy  enlisted  man  in  the  other  experience-rate  groups  should  be 
paid  in  comparison  to  the  group  to  which  he  assigned  the  maximum  bonus  pay 
of  $100. 

The  bonus  pay  estimates  were  analyzed  and  aggregated  in  the  same  way  as 
the  base  pay  estimates.  The  resultant  summary  tables,  however,  were  quite 
different  from  the  base  pay  summary  tables,  and  the  LOS-dependent  utility 
curves  did  not  show  the  linearity.  More  important,  the  general  shape  of 
these  bonus  pay  utility  curves  were  similar  to  that  of  the  utility  curves 
developed  in  the  Delphi  experiment.  These  findings  support  the  assertion 
that  base  pay  is  a biased  proxy  for  utility. 
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It  should  be  noted  that  "bonus  pay"  was  defined  in  the  questionnaire 
as  entirely  separate  and  distinct  from  any  existing  pay  supplements  such 
as  proficiency  pay,  hazard  duty  pay,  or  reenlistment  pay.  Although  this 
distinction  was  made,  many  of  the  participants  made  clear,  in  their 
written  comments  and  in  conversation  with  those  administering  the  question- 
naire, their  opposition  to  the  proficiency  pay  system,  in  particular,  and 
to  any  bonus  pay  system  in  general. 

The  third  proxy  for  value,  achievement,  was  incorporated  in  the  question- 
naire to  solicit  opinions  regarding  the  possible  relationship  between  the 
overall  value  of  the  individual  to  the  Navy  and  the  degree  to  which  he 
possesses  tiie  attributes  which  contribute  to  that  overall  value.  In  develop- 
ment of  this  question,  it  was  assumed  that  the  value  of  an  average  Navy 
enlisted  man  changes  in  some  orderly  fashion  as  he  advances  in  length  of 
service  and  rate,  and  that  this  may  be  due  to  changes  in  his  exercise  of 
the  following  three  attributes: 

1.  Military  Ability 
, 2.  Leadership  Ability 

3.  Professional/Vocational  Ability 

Commonly  accepted  definitions  of  leadership  ability  and  professional  ability 
were  supplied  to  the  participants.  Military  ability  was  defined  as  "the 
ability  of  the  enlisted  man  to  respond  to  command  in  an  emergency  situation." 

To  solicit  achievement  estimates  from  the  participants,  two  questions 
were  developed.  The  first  dealt  with  the  importance  of  each  of  the  three 
attributes  selected  to  successful  performance.  The  matrix  used  to  record 
these  estimates  is  shown  in  Figure  4.  The  participant  was  asked  to  record 
a value  of  100  to  identify  what  he  considered  to  be  the  most  important 
attribute  and  to  record  for  the  remaining  two  attributes  a value  correspond- 
ing to  his  estimates  of  their  relative  importance.  The  second  question 
directed  the  participant  to  estimate  the  achievement  of  each  of  these 
attributes  by  the  average  Navy  enlisted  man  as  he  gained  in  experience. 

The  matrix  in  Figure  5 was  used  to  record  these  estimates.  The  participant 
was  directed  to  record  a value  of  100  to  identify  the  experience  group  in 
which  he  considered  the  average  Navy  enlisted  man  reaches  maximum  achieve- 
ment of  the  attribute  and  to  record,  for  each  of  the  other  experience 
groups,  values  which  correspond  to  his  estimates  of  the  reduced  achievement. 
Processing  and  analysis  of  these  two  tabulations  were  directed  toward  map- 
ping these  measures  into  a composite  achievement  measure  as  a function  of 
both  rate  and  experience. 
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Figure  4.  Matrix  for  estimating  relative  importance  of 
selected  attributes  to  successful  job  performance. 


Figure  5.  Matrix  for  estimating  achievement  of  selected 
attributes  by  the  average  enlisted  man  during  his  career. 


Aggregate  utility  tables  derived  from  the  three  proxies — base  pay, 
bonus  pay,  and  achievement  estimates  for  the  overall  Navy — are  shown  in 
Figure  6.  LOS-dependent  utility  estimates  derived  from  the  summary  tables 
for  each  of  the  five  rating  groups  are  shown  in  Figures  7 through  9.  From 
Figure  7,  it  can  be  seen  that  the  base  pay  estimates  obtained  from  the 
second  experiment  again  conform  with  the  existing  base  pay  table.  The 
curves  show  nearly  linear  growth  in  utility  from  the  start  of  the  enlisted 
man's  career  to  the  end  at  year  30.  Thus,  it  is  apparent  that  this  question 
inherently  involves  a considerable  amount  of  bias  as  to  what  the  base  pay 
is  as  opposed  to  what  it  "should"  be. 
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Figure  b.  Tabular  results  from  the  Broadcast  experiment. 
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Comparison  ol  Delphi  I and  Broadcast:  Bxper  intents 


In  analyzing  the  results  of  the  Delphi  1 and  the  broadcast  experiments, 
little  statistical  difference  was  found  in  the  utility  estimates  as  a 
function  of  skill  group.  Thus,  for  the  purpose  of  comparing  the  results, 
the  utility  estimates  for  the  overall  Navy  enlisted  man  will  suffice.  A 
graphic  presentation  of  these  results  is  shown  in  Figure  10,  which  plots 
the  utility  curves  constructed  from  the  aggregate  bonus  pay  utility  table 
and  the  aggregate  achievement  table  from  the  Broadcast  experiment  against 
the  mean  and  interquartile  utility  curves  from  the  Delphi  experiment. 

At  once,  it  is  readily  apparent  that  the  results,  from  experiment  to 
experiment,  are  different.  The  Broadcast  validation  experiment  curves 
show  a steeper  and  earlier  rise  to  the  maximum  utility  and  a pronounced 
decline  in  utility  in  the  later  years  of  the  career.  The  question  is 
whether  these  differences  are  sufficient  evidence  to  invalidate  the  utility 
concept  or  the  utility  estimates  collected  from  the  Delphi  1 experiment. 

As  indicated  previously,  the  purpose  of  the  Broadcast  experiment  was 
to  test  the  validity  of  the  Delphi  I experiment  results.  It  was  designed 
to  enable  an  assessment  of  comparability  and  consistency  of  the  utility 
estimates  botli  within  and  between  experiments,  rather  than  to  achieve  a 
repetition  of  results.  This  is  clear  from  the  questionnaire  developed  and 
administered  in  the  second  experiment,  which  avoided  the  utility  concept 
by  having  the  participants  record  estimates  for  a variety  of  proxies.  In 
addition,  estimates  were  solicited  at  a higher  level  of  aggregation,  for 
each  of  four  experience  classes,  and  for  each  of  three  rate  groups,  rather 
than  for  each  year  of  service  and  paygrade.  To  respond  properly  to  the 
question  of  validity,  then,  the  consistency  and  comparability  of  the 
results  must  be  examined  in  the  light  of  the  deliberate  differences  in 
the  instruments  used  in  each  experiment. 

First,  the  question  of  consistency  will  be  addressed.  In  successive 
rounds  of  the  Delphi  I experiment,  the  estimates  of  botli  panels  converged, 
independently,  to  a set  of  utility  curves  which  are  quite  similar  to  the 
eye  and,  for  the  most  part,  statistically.  Only  the  distinction  between 
the  senior  officer  panel  and  the  junior  officer  panel  is  statistically 
significant,  and,  even  in  this  case,  no  serious  difference  between  these 
curves  is  apparent.  In  analysis  of  the  Broadcast  experiment,  no  statistically 
significant  variation  was  found  from  panel  to  panel  or  from  rating  group  to 
rating  group.  Only  the  variation  due  to  the  base  pay  question  is  statistically 
significant.  Further,  the  summary  utility  curves  developed  from  two 
independent  estimates  in  each  experiment  have  the  same  general  shape. 

Within  reasonable  limits,  the  criteria  for  consistency  are  satisfied  over 
most  of  the  career  length. 
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This  evidence  that  the  part ic ipants  in  each  experiment  were  able  to 
evaluate  the  utility  of  personnel  consistently  tends  to  confirm  the 
validity  of  the  utility  concept.  The  remaining  question  of  comparability 
depends  on  the  selection  of  numeric  values  to  be  assigned  in  applying  the 
utility  concept  to  manpower  planning,  and  on  the  method  of  comparison. 
Since  the  Broadcast  experiment  estimates  were  solicited  at  a less  detailed 
level,  an  intermediate  transformation  is  necessary  to  place  the  estimates 
on  the  same  basis.  Thus,  the  second  experiment  utility  tables  were  trans- 
formed into  an  LOS-dependent  utility  curve  (B-K  Dynamics,  1973)  by  using 
the  statistics  describing  the  existing  enlisted  population  distribution 
in  conjunction  with  the  following  equivalents  for  the  rate  and  experience 
factors : 


Kale  ( iroups 


Equivalent  I’aygrade 


Apprentice 


E-3  ami  I >4 


L 


Journeyman 

Supervisor 

I'.xperience  (iroups 
1st  Termer 
2nd  Termer 
Less  than  20  years 
20  30  years 


E-5  and  E-6 
L-7  through  E-9 

Equivalent  Length  of  Service  (LOS) 
1 4 years 

5-9  years 
10-20  years 
21  30  years 


The  equivalent  paygrades  for  the  rate  groups  were  based  on  definitions 
given  to  the  participants  in  the  second  experiment.  The  equivalent  years 
of  service  for  the  experience  groups  were  obtained  by  averaging  the  length 
of  service  estimates  for  the  1st  and  2nd  termer  solicited  from  the  partici- 
pants. 

The  utility  estimates  for  LOS  1 through  30  were  computed  from  the 
tabular  estimates  for  each  rate  group  in  the  experience  group  containing 
the  LOS.  The  computation  used  the  actual  population  proportions  of 
apprentices,  journeymen,  and  supervisors  (as  defined  by  their  equivalents) 
to  develop  a weighted  average  utility  estimate  at  each  LOS. 

The  essential  point  to  be  realized  from  this  brief  description  is  that 
the  shape  of  the  constructed  utility  curves  depends  not  only  on  the  partici- 
pant's estimates,  but  also  on  the  population  statistics.  At  the  extreme. 
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one  could  assert  that  the  transformation  of  a participant’s  tabular 
estimates  into  LOS-dependent  estimates  can  be  valid  only  if  the  par- 
ticipant's own  estimate  of  the  population  mix  is  used.  At  the  other 
extreme,  the  curves  could,  of  course,  be  made  more  nearly  the  same  by. 
using  a carefully  selected  or  constructed  population  mix.  As  a practical 
matter,  it  appears  reasonable  to  use  the  existing  population  mix. 

In  examining  the  LOS-dependent  utility  estimates,  it  is  seen  that, 
for  the  first  5 or  so  years  of  the  career,  the  mean  utility  curve  from 
the  first  experiment  is  approximately  midway  between  the  bonus  pay  and 
achievement  utility  estimates.  Over  the  5 to  20  year  span,  however,  both 
utility  estimates  from  the  Broadcast  experiment  are  consistently  higher 
than  the  mean  utility  curve.  Some  of  this  disparity  could  be  due  to  bias 
introduced  by  the  transformation  technique  which  produced  estimates  that 
were  also  consistently  higher  than  the  LOS-dependent  utility  curves  in  the 
mid-career  years.  Some  of  this  effect  was  attributed  to  the  participants' 
misconception  of  the  true  population  mix.  The  remaining  disparity  could  be 
due  to  bias  introduced  by  the  transformation  technique  or  by  the  population 
statistics.  This  bias  could  be  operative  in  the  transformation  of  the 
second  experiment  utility  estimates. 

Over  the  20  to  30  year  span,  the  differences  are  quite  distinct  and 
irreconcilable.  The  participants  in  the  second  experiment,  within  the 
framework  of  the  proxies  used  for  utility,  see  a much  more  pronounced 
decline  in  the  usefulness  of  the  average  Navy  enlisted  man.  The  decline 
in  utility  in  the  fourth  experience  group  is  also  quite  evident  in  the 
tabular  presentation.  Whether  this  drop  in  utility  as  seen  by  the  par- 
ticipants in  the  second  experiment  is  true  or  not  is  an  important  issue 
for  the  Navy.  However,  for  our  purposes,  these  distinct  differences  may 
not  be  of  overall  significance  in  their  intended  application  in  the  utility 
model  due  to  the  relatively  smaller  numbers  of  enlisted  men  with  more  than 
20  years  of  service. 

It  is  believed  that  the  limitations  to  generating  comparable  numeric 
estimates,  as  discussed  above,  are  inherent  with  any  transformation  technique. 
For  this  reason,  the  apparent  differences  between  experimental  results  should 
be  given  less  weight  than  the  apparent  consistency  of  results  within  the 
experiments.  This  factor,  together  with  the  intended  application  of  the 
data,  argues  against  rejection  of  the  utility  concept. 

The  cost  and  personnel  inventory  strength  data  with  which  the  utility 
estimates  are  to  be  compatible  are  in  a 9-grade  by  31-year  format.  Since 
the  utility  curves  describe  the  change  in  usefulness  of  the  average  enlisted 
man  progressing  through  the  system,  the  utility  curves  supply  only  certain 
elements  of  the  9 x 31  matrix  (Figure  11).  Completion  of  the  matrix  for 
LOS-paygrade  cells,  other  than  those  of  the  average  enlisted  man,  requires 
estimates  of  the  utility  of  the  enlisted  man  whose  pattern  of  advancement 
is  different  from  the  average.  One  way  of  posing  the  problem  is  to  examine 
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the  nut  t r ix . II  le  pattern  of  e lion  ge  could  lu'  approximated  hy  usiii^  *i  lop  — 
learning  curve,  or  hy  employ  in;;  the  tabular  estimates  in  the  second  experi- 
ment which  show  the  relative  change  in  utility  of  each  given  rate  group 
with  increasing  experience.  To  respond  to  this  problem  and  to  further 
explore  the  personnel  utility  concept,  a third  experiment  was  conducted. 

PAV  GHAOES 


Figure  I I . Career  path  of  the  average  progressor. 


Delphi  II  Experiment 


After  completion  of  the  broadcast  experiment,  a prototype  utility 
model  was  implemented  as  part  of  the  ADSTAI’  personnel  policy  planning  and 
evaluation  system.  This  utility  model  translated  the  Delphi  I average 
progressor  utility  curve  into  paygradc  utiliLy  curves.  This  particular 
method  was  chosen  from  a number  of  alternatives,  the  choice  turning,  on 
the  definitions  of  the  average  progressor  and  the  average  man  serving, 
and  certain  assumptions  about  the  paygradc  utility  curves  (e.g.,  no  over- 
lap between  grades;  no  K-5  can  be  worth  more  than  an  E-h). 
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While  the  prototype  utility  model  was  deemed  sufficient  to  support 
development  and  testing  of  an  optimization  methodology  and  to  provide 
another  factor  in  evaluation  of  personnel  policies,  there  was  still 
considerable  uncertainty  with  regard  to  the  paygrade  utility  estimates. 

In  addition,  the  disparities  between  the  utility  estimates  from  each  ex- 
periment, particularly  in  characterizing  the  drop  in  utility  of  enlisted 
men  near  the  end  of  their  career,  prompted  further  research  into  this 
assessment . 

To  resolve  these  questions,  a third  experiment,  Delphi  II,  was  initiated. 
Since  the  previous  two  experiments  had  involved  officers  and  senior  enlisted 
men  serving  in  a shore  tour,  this  experiment  was  addressed  only  to  personnel 
serving  on  operational  billets  in  the  Fleet.  In  all,  some  75  officers  and 
enlisted  men  (making  up  three  panels)  serving  in  the  2nd  Fleet  participated 
in  the  third  experiment. 

The  questionnaire  for  Delphi  II  (Appendix  A)  was  designed  to  address 
the  particular  questions  raised  as  a result  of  the  first  two  experiments. 
Because  of  the  clear  advantage  of  the  Delphi  procedures  over  broadcast 
one-shot  questionnaires,  a Delphi  exercise  was  chosen  as  the  vehicle  for 
the  third  experiment. 

In  the  previous  experiments,  both  graphic  and  tabular  estimates  of 
personnel  utility  had  been  solicited.  Since  results  of  the  first  experi- 
ment showed  that  estimates  obtained  from  the  two  methods  were  comparable, 
there  was  no  need  to  choose  between  methods.  There  was,  however,  a need 
in  the  second  experiment  to  isolate  certain  questions  about  the  utility  of 
an  enlisted  man  that  suggested  that  the  following  characteristics  of  the 
utility  curve  should  be  addressed  separately: 

1.  The  slope — the  rate  (growth)  at  which  the  average  enlisted  man 
attains  usefulness  to  the  Navy. 

2.  The  peak — the  year  of  service  at  which  the  average  enlisted  man 
reaches  his  peak  usefulness. 

3.  The  late  years — the  change,  if  any,  in  usefulness  of  the  average 
enlisted  man  beyond  the  years  after  he  reaches  his  peak. 

The  questionnaire  developed  for  the  first  round  of  the  Delphi  II  experi- 
ment was  organized  into  four  parts.  Part  1 was  designed  to  solicit  estimates 
which  could  be  compared  to  the  results  of  the  two  previous  experiments.  The 
following  five  questions  were  posed  regarding  the  utility  of  the  average 
enlisted  man  progressing  in  paygrade  and  length  of  service: 

1.  At  what  year  of  service  does  the  average  progressor  reach  his 
maximum  value  to  the  Navy? 

2.  Until  what  year  of  service  does  he  maintain  his  maximum  value? 
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3.  If  the  answer  to  question  2 is  less  than  30,  what  is  his  value  to 
t lie  Navy  in  the  30th  year  (as  a % of  his  maximum  value)? 

4.  How  much  of  his  maximum  value  has  he  obtained  by  the  time  he 
reaches  the  end  of  his  first  enlistment  (as  a % of  his  maximum  value)? 

5.  In  what  year  of  service  does  he  reach  50%  of  his  maximum  value? 

Part  2 solicited  estimates  regarding  the  utility  of  an  average  enlisted 
man  as  he  progresses  in  years  of  service  while  remaining  at  a specific 
paygrade.  For  each  paygrade,  E-3  through  K-9,  the  participants  were  asked 
to  supply  the  following  three  estimates: 

1.  At  what  year  of  service  does  a man  reach  his  maximum  value  in  this 
paygrade  (year  from  1 to  30)? 

2.  Assume  the  man  remains  in  the  paygrade  throughout  the  remainder  of 
his  30-year  career.  Until  what  year  of  service  does  a man  in  this  paygrade 
maintain  his  maximum  value  (year  from  1 to  30)? 

3.  If  the  answer  to  question  2 is  less  than  30,  what  is  his  value  to 
the  Navy  at  30  years  (as  a percentage  of  his  maximum  value  at  this  paygrade)? 

Part  3 solicited  estimates  of  the  maximum  value  attained  by  the  average 
enlisted  man  at  a paygrade  in  relation  to  all  others  paygrades.  To  set  the 
scale,  the  assumption  was  made  that  the  maximum  value  attained  by  the  E-9 
is  100. 

Part  4 solicited  estimates  of  the  value  of  the  average  enlisted  man 
upon  entrance  to  the  Navy.  As  a proxy,  the  participants  were  asked  to 
estimate  on  a scale  ranging  from  0 to  100  the  value  of  Direct  Procurement 
Petty  Officers  (DPPOs)  at  advanced  paygrades  E-2  through  E-9. 

Analyses  of  the  results  and  rounds  2,  3,  and  4 of  the  experiment  pro- 
ceeded in  accordance  with  conventional  Delphi  procedures.  Means  anil  inter- 
quar tiles  were  computed  and  recycled  back  to  the  participants  for  reevalua- 
tion. Requests  for  additional  information  were  honored  and  rationale 
statements  and  defenses  were  abstracted  and  circulated. 


Summary  results  from  the  first  round  of  Delphi  II  are  illustrated  in 
Figure  12,  Table  1,  and  Figure  13.  For  Figure  12,  the  responses  of  all 
the  participants  to  Part  1 of  the  questionnaire  were  averaged,  and  the 
resultant  average  progressor  curve  (tent)  was  plotted  against  Delphi  I 
and  Broadcast  questionnaire  results.  The  maximum  value  of  the  average 
progressor  curve  falls  within  the  interquart  lies  of  the  Delphi  I curves, 
while  the  career-end  portion  corresponds  more  closely  witli  the  broadcast 
experiment  estimates.  (See  Appendix  B for  Summary  Statistics,  Delphi  11 
Rounds  1,  2,  and  3). 
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Comparison  ol  Paygrade  Utility 

TABl  l 
Estimates 

1 

tiom  Delphi  l and  II  1 \poiuiienls 

Paygrade 

IVIplu  1 

IVIpIn  II,  Round  1 

E-l 

8 

15 

E-2 

17 

">  •> 

E-3 

29 

32 

t-4 

49 

48 

E-5 

67 

64 

E-6 

83 

77 

E-7 

94 

89 

E-8 

98 

94 

E-9 

96 

99 

In  Table  1,  the  paygrade  utility  estimates  from  Part  3 of  Delphi  II 
Round  1 are  compared  with  the  corresponding  estimates  from  Delphi  I.  At 
paygrade  E-4  and  above,  the  estimates  are,  at  most,  six  percentage  points 
different . 

In  Figure  13,  the  paygrade  utility  curves  (tents)  derived  from  Parts 
2,  3,  and  4 of  the  Delphi  II  questionnaire  are  plotted  against  the  results 
of  the  previous  experiments.  At  LOS  = 0,  the  average  utility  estimates 
characterizing  the  Direct  Procurement  Petty  Officers  are  plotted  for  each 
paygrade.  The  maximum  values  were  obtained  from  the  averages  of  Part  3; 
these  maximum  values  were  combined  with  the  last  question  of  Part  2 to 
compute  the  values  plotted  at  LOS  = 30. 

As  may  be  seen  in  results  of  both  Delphi  experiments,  the  average 
enlisted  progressor  reaches  a point  of  maximum  utility  at  approximately 
24-27  years  of  service  and  thereafter  declines  in  utility.  This  general 
pattern  also  applies  to  the  average  enlisted  man  at  each  paygrade.  Those 
who  do  not  get  promoted  (i.e.,  those  who  remain  ir.  grade)  decline  in  value 
to  the  Navy  at  advanced  years  of  service. 

Even  at  paygrades  E-8  and  E-9,  this  decline  at  advanced  LOS  was  observed 
by  the  participants  in  the  Delphi  experiment.  Interpretation  of  these 
results  raises  several  questions,  such  as: 

1.  Does  this  simply  describe  "short-timer"  behavior,  which  is  not  a 
function  of  LOS,  but,  rather,  is  related  to  the  approacning  end  of  career? 

2.  Is  this  evidence  that  E-3's  and  E-9's  are  improperly  and  inef- 
fectively used  by  the  Navy? 
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Figure  12.  Comparison  of  Delphi  II  Round  1 average  progressor  tent 
and  results  of  Delphi  I and  Broadcast  experiments. 
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tents  uml  results  ol  previous  experiments 


Tabulations  of  these  results  were  returned  to  the  participants  for 
reevaluation  in  the  second  round.  The  averages  over  all  responses 
changed  very  little  for  the  most  part,  while  the  spread  measured  by  the 
Intcripiarl  lie  responses  to  each  quest  lou  nail  owe. I considerably.  I‘ait 
4 of  the  questionnaire  dealing  with  I lie  value  ol  Dl'I’o'a  upon  nil  i y lo 
the  Navy  showed  the  greatest  spread.  In  this  regard,  it  was  evident,  that 
few  participants  had  ever  had  an  opportunity  Lo  see,  let  alone  evaluate, 
the  value  of  a DPPO.  In  fact,  since  this  section  of  the  utility  curve 
applies  to  such  a small  population,  it  was  included  in  the  experiment 
only  to  complete  the  curves  in  the  early  LOS  cells.  Administration  did 
not  expect  to  derive  reliable  yardsticks  for  evaluating  DPPO  options. 

Table  2 shows  the  means  establishing  the  paygrade  utility  curves  and 
957.  confidence  intervals  about  the  means.  Figure  14  displays  the  third 
round  estimates  for  comparison  with  the  early  experiments. 

Another  means  of  evaluating  the  various  estimates  for  the  average 
progressor  utility  curve  was  initiated  on  the  third  round  of  Delphi  II. 
The  participants  were  furnished  the  tent-shaped  paygrade  utility  graphs 
and  asked  to  review  and  redraw  them  to  reflect  their  assessment.  Their 
attention  was  particularly  directed  to  the  overlapping  in  utility  from 
paygrade  to  paygrade  and  to  the  decline  in  value  after  passing  peak 
utility. 


TABLI  2. 

Delphi  II.  Round  1.  Means  and  Confidence  Limits  Paygrade  Utility  Curves 


Paygrade 

Value 
at  Paltry 

Maximum 

Value 

Value  at 
Lind  of  Career 

LOS  Reaching 
Maximum  Value 

LOS  Maintaining 
Maximum  Value 

Means  Over  All  Panels 

lid 

1 4.X 

d 1 .d 

4.4 

2.6 

7.d 

1-4 

24.7 

4(>.6 

IX. 7 

4.d 

4.6 

li-5 

d6.6 

64.7 

d4.7 

7.1 

Id.d 

1-0 

45.0 

77.4 

50.0 

10.4 

IX  1 

1-7 

51.2 

XX. 5 

(>(.4 

14  4 

217 

1 -X 

55.5 

4d.4 

77.5 

17  4 

25.1 

1-4 

5‘>.5 

44.1 

X5.6 

14. X 

27. d 

4.5'<  Confidence  Limits  About  the  Mean 

Id 

Id. 2.  10.4 

dO. 2.  d2.4 

7.4.  12.4 

2.4.  2.X 

5.5.4  1 

14 

2d.0.  26.4 

45.7.47.5 

15  X.  21.6 

4.1. 4.5 

S.d.  10  4 

1-5 

d4  d.  dX.4 

62.4.  (>(i.5 

d0.4.  d‘».() 

6.X.  7.4 

U 4. 14  2 

1-0 

42.X.  47.2 

76.2,  7X.6 

45. d.  54.7 

10  2.  10.6 

17  5,  ix. 7 

1-7 

4X.4,  5d.‘> 

X7.X.  84,2 

61.7.  71.1 

14.0.  14  X 

21  1.  22. d 

1 -X 

52.X.  5X.2 

‘>d.d.  44.5 

7d.7,  XI  d 

16.4.  17.4 

24.5.  25.7 

lid 

56.6.  62.4 

48.7.  44.5 

XI. 7.  80.5 

|4.|.  20.5 

26.4.  27.7 

2d 
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Finally,  given  that  the  curvon  del  Inc  the  change  in  utility  of  the 
uvernge  man  by  paygrnde,  the  par  1 1 c Ipunt  h were  naked  to  mark  on  each 
curve  the  position  of  the  averuge  man  who  has  )uat  been  advanced  to  grade, 
in  Figure  15,  the  resultant  mean  and  Interquartile  average  progressor 
utility  curves  are  plotted  against  results  of  the  previous  experiments. 

As  seen,  the  exact  location  of  the  average  progressor  utility  curve  varies 
depending  on  the  instrument  or  type  of  question,  but  is  generally  similar 
over  all  three  experiments. 

Analysis  of  the  responses  to  the  fourth  and  final  round  of  the  experi- 
ment should  minimize  change  in  the  averages  from  the  previous  round.  As 
seen  in  Table  3,  the  largest  percentage  change  observed  (4%)  was  the  utility 
value  of  an  E-5  at  LOS  30,  which  changed  in  value  from  34.7  to  33.3. 


TABLE  3. 

Comparison  of  Delphi  II  Round  4 and  Delphi  II  Round  3 Results 


Paygrade 

Round 

Value 
at  entry 

L.OS 

Reaching 

Maximum 

LOS 

Maintaining 

Maximum 

Value  at 
M aximum 

Value  at  Laid 
of  Career 

E-3 

4 

No  change 

2.8 

7.1 

32.7 

10.1 

3 

14.8 

2.6 

7.3 

31.3 

9.9 

L-5 

4 

No  change 

7.1 

13.5 

65.1 

33.3 

3 

36.6 

7.1 

13.3 

64.7 

34.7 

E-7 

4 

No  change 

14.0 

22.2 

89.0 

68.6 

3 

51.2 

14.4 

21.7 

88.5 

66.4 

30 


CONCLUSIONS 


From  the  consistency  of  the  results  in  both  Delphi  experiments,  it 
can  be  concluded  that  the  Delphi  technique  seems  to  be  a valid  technique 
to  measure  the  utility  of  the  enlisted  man  by  paygrade  and  LOS. 

Since  an  enlisted  man's  usefulness  decreases  if  he  remains  in  the 
same  paygrade  (i.e.,  does  not  get  promoted)  after  a period  of  time, 
the  Navy  could  possibly  benefit  by  employing  "early  out"  policies  for 
enlisted  men  with  no  chance  of  promotion. 


RECOMMENDATIONS 

The  results  of  this  study  should  be  used  to  develop  a utility  model 
to  help  the  Bureau  of  Naval  Personnel  define  long-range  management  goals 
for  the  Navy  Personnel  enlisted  force.  The  utility  model  could  be  used 
along  with  other  models  such  as  cost,  inventory,  and  reenlistment 
elasticity  models  to  optimize  personnel  force  distributions  by  skill 
grouping,  length  of  service,  and  paygrade. 
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DEPARTMENT  OF  THE  NAVY 

BUREAU  OF  NAVAL  PERSONNEL 

WASHINGTON.  D C 2037 0 


Pers-Ax/c lw 
Ser  398/73 


MEMORANDUM  FOR  PARTICIPANTS  IN  DELPHI  EXPERIMENT 
Subj : DELPHI  Determination  of  Utility 

Enel:  (1)  Test  Questionnaire 

1.  In  recent  months  a considerable  amount  of  research  has  been  done  in 
general  utility  theory  as  applied  to  military  manpower.  One  of  the  out- 
growths of  this  research  is  an  attempt  to  use  the  DELPHI  survey  method 
to  determine  the  utility  to  the  Navy  of  enlisted  men  as  they  progress  in 
both  length  of  service  and  paygrade.  The  attached  questionnaire  is  one 
test  vehicle  designed  to  get  at  this  very  difficult  concept. 

2.  You  have  been  selected  as  one  member  of  a panel  to  evaluate  this  con- 
cept. In  the  DELPHI  methodology,  two  points  are  paramount:  first,  itera- 
tion of  results  and  second,  anonymity.  You  will  receive  a second  question- 
naire, and  perhaps  a third  and  fourth  later,  in  which  the  results  of  pre- 
vious questionnaires  are  fed  back  to  vo\i  with  requested  additional  informa- 
tion and  a review  of  comments  made  by  others.  None  of  the  panel  members 
will  be  cited  by  name,  so  you  will  be  required  to  review  the  results  based 
on  the  overall  group  view,  not  the  relative  ranks  or  authority  of  individual 
respondents.  You  may  know  some  or  most  of  the  other  respondents  in  this 
experiment.  Some  of  you  may  even  be  in  the  same  unit.  Therefore,  please 
preserve  the  necessary  anonymity  by  not  discussing  this  experiment  with 
others  until  the  experiment  is  completed. 

3.  On  the  blank  sheet  attached  to  the  questionnaire,  please  indicate  to 
the  extent  you  can  (1)  any  general  assumptions  or  rationale  behind  your 
answers , and  (2)  any  request  for  additional  information  which,  if  avail- 
able, you  think  would  help  you  make  more  valid  judgments  about  these 
questions . 

4.  The  concept  of  the  utility  to  the  Navy  of  the  average  man,  during  a 
thirty  year  career,  is  one  which  requires  some  new  thinking  about  en- 
listed men  which  has  not  been  required  in  your  previous  tours.  On  the 
other  hand,  there  is  no  wav  this  question  can  be  answered  without  drawing 
on  the  actual  experience  you  have  had  in  leading  enlisted  men  in  the  Navv 
environment.  The  following  points  cannot  define  in  detail  the  concept 

in  which  we  are  interested;  to  do  so  might  drive  the  answers,  which  we 
don't  want  to  do.  These  points  may,  however,  help  you  to  clarify  the  con- 
cept in  vour  own  mind.  Most  of  the  questions  refer  to  the  average  sailor 
as  he  progresses  from  F.1  to  FQ  in  a thirty  year  career.  Fveryone  has  two 
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Subj:  DELPHI  Determination  of  Utility 

"average  sailors"  in  mind;  one,  the  average  of  the  people  who  have  worked 
for  him  and  two,  his  own  idea  of  what  an  average  sailor.  Navy-wide,  is. 

These  are  usually  not  the  same.  In  this  questionnaire,  we  want  you  to 
respond  based  on  the  people  you  have  known  and  with  whom  you  have  worked. 

The  Navy-wide  average  man  should  then  develop  later  as  a result  of  the 
integration  of  all  the  questionnaires.  It  is  very  difficult  to  separate 
the  ideas  of  utility  to  the  Navy,  in  general,  and  utility  in  a specific 
job.  The  Navy  pays  an  E-7  a great  deal  more  than  it  pays  to  an  E2.  In- 
tuitively, the  Navy  must  believe  that  the  E7  is  worth  more.  If,  however, 
you  try  to  tie  the  concept  to  a specific  billet,  the  concept  becomes  very 
ttoublesome.  Obviously,  it  is  very  hard  to  decide  who  would  be  most  use- 
ful for  chipping  paint,  an  E2  or  an  E7.  Tf  possible,  try  to  take  the 
broader  view  of  the  "general  good".  One  way  to  look  at  it,  which  may  be 
helpful  to  you,  is  to  try  to  answer  this  question:  "Paygrade  not  withstand- 
ing, in  an  emergency  situation,  how  much  of  the  total  rating  work  would  I 
trust  a man  of  a specific  paygrade  and  experience  level  (length  of  service) 
to  do  for  me?"  for  most  naval  officers,  this  would  result  in  ascribing 
• higher  value  to  a chief  than  to  a striker. 

5.  If  you  would  like  to  discuss  the  concept  personally  before  we  are 
through,  please  call  Bob  I,ehto,  Pers-Ax  (Autovmn)  224-4491.  When  you  have 
completed  this  questionnaire,  please  mail  in  enclosed  envelope.  If  pos- 
sible, attempt  to  complete  the  questionnaire  and  return  it  in  a week  to 
ten  days. 


R.K.  LEHTO 

Special  Assistant  for 
Enlisted  Force  Analysis 
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A.  Consider  first  the  average  Navyman;  the  man  who  completes  a thirty 
year  career  and  progresses  normally  through  all  the  paygrades  from  El  to 
E9.  We  make  the  following  general  assumptions  about  this  man's  value  or 
usefulness  to  the  Navy.  The  average  recruit  is  assumed  to  enter  the  Navy 
with  zero  value.  This  doesn't  mean  we  think  recruits  are  worth  nothing, 
but  rather  that  we  are  trying  to  measure  the  added  (or  accrual)  useful- 
ness that  a man  gains  while  he  is  in  the  Navy.  We  further  assume  that 
as  a man  is  trained  and  gains  experience,  his  usefulness  to  the  Navy  in- 
creases. At  some  point  in  his  thirty  year  career,  the  average  progressor 
will  reach  a point  of  maximum  value  to  the  Navy  which  we  will  define  here 
as  1 00  or  100Z,  whichever  nomenclature  you  prefer.  The  following  ques- 
tions solicit  your  views  as  to  how  this  man's  value  or  usefulness  to  the 
Navy  changes  as  he  progresses.  In  general,  the  questions  ask  that  you 
make  judgments  in  relation  to  the  maximum  value  of  100  defined  above. 

1.  At  what  year  of  service  does  the  average  progressor  reach  his 
maximum  value  to  the  Navy? 

th  year  of  service 

2.  Until  what  year  of  service  does  he  maintain  this  maximum  value? 
th  year  of  service 

3.  If  your  answer  to  question  2 was  less  than  30,  what  is  his  value 
to  the  Navy  in  the  30th  year? 

% of  his  maximum  value 
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4.  How  much  of  his  maximum  value  has  he  obtained  by  the  time  he 
reaches  the  end  of  his  first  enlistment?  (Assume  a first  enlistment 
length  of  four  years.) 

% of  his  maximum  value 

5.  In  what  year  of  service  does  he  reach  50%  of  his  maximum  value? 
th  year  of  service 

B.  The  questions  in  A above  dealt  with  the  value  of  the  average  Navyman 
who  progresses  in  paygrade  through  his  thirty  year  career.  This  next 
question  requires  information  about  an  average  man  as  he  progresses  through 
a specific  paygrade.  A table  is  provided  below  for  your  answers.  The 
three  questions  are  repeated  here  with  the  keys  to  the  table.  The  years 
should  be  specified  on  a basis  of  total  career  length  of  service  (1-30). 

"MAXIMUM  YEAR" — At  what  year  of  service  does  a man  reach  his  maximum 
value  in  this  paygrade?  (Year  from  1-30) 

"MAINTAIN  YEAR" — Assume  the  man  remains  in  the  paygrade  throughout 
the  remainder  of  his  30  year  career.  Until  what  year  of  service  does  a 
man  in  this  paygrade  maintain  his  maximum  value?  (Year  from  1-30) 

"VALUE  AT  30TH  YEAR"— If  your  answer  in  the  MAINTAIN  year  column  was 
less  than  30,  what  is  his  value  to  the  Navy  at  30  years  (as  a percentage 
of  his  maximum  value  at  this  paygrade) ? 
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E-3 
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E-5 


E-7 


E-8 


E-9 


MAXIMUM  YEAH 


th 


th 


th 


th 


th 


th 


th 


MAINTAIN  YEAR 


th 


th 


th 


th 


th 


th 


th 


VALUE 

AT  30TH  YEAR 


% 


% 


% 


% 


1 


% 
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C.  The  previous  questions  have  concerned  themselves  with  the  change  over 
time  of  value  of  the  average  enlisted  man.  This  next  question  deals  with 
his  value  at  a paygrade  in  relation  to  all  other  paygrades.  Indicate  in 
the  table  below  on  a scale  of  0-100  the  average  value  of  the  average  pro- 
gressor  at  each  paygrade. 

VALUE  OF  AVERAGE  PROGRESSOR 

E-l  

E-2  

E-3  

E-4  

E-5  

E-6  

E-7  

E-8  

E-9 
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D.  In  the  past  the  Navy  has,  at  one  time  or  another,  recruited  men  at 
advanced  paygrades  based  on  their  civilian  experience.  Some  of  these  pro- 
grams have  been  called  Direct  Procurement  Petty  Officer  (DPPO)  programs. 

In  general,  these  programs  have  assumed  that  the  man  had  the  technical 
qualifications  for  the  paygrade  and  could  learn  the  "Navy  way"  on  the  job. 
Please  indicate  below  your  judgments  as  to  the  value  (on  a scale  of  0-10C) 
of  such  men  at  the  time  of  recruitment.  In  other  words,  we  are  asking 
for  judgments  about  the  value  of  a man  at  a specific  paygrade  but  with 
zero  years  of  service. 


E-l 


VALUE 


E-2 


E-3 


E-4 


E-5 


E-6 

E-7 

E-8 


E-9 
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DEPARTMENT  OF  THE  NAVY 

BUREAU  OF  NAVAL  PERSONNEL 
WASHINGTON  DC  20370 
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IN  REPLY  REFER  TO 

Pers-2X/mmo 
Ser  717/73 

aus  1 0 m 


MEMORANDUM  FOR  PARTICIPANTS  IN  DELPHI  EXPERIMENT 

Subj : DELPHI  Determination  of  Utility;  second  questionnaire 

Enel:  (l)  Additional  Discussion  of  Experiment 

(2^  Your  First  Questionnaire  (not  included) 

(3)  Averages  of  Response  to  First  Questionnaire  (not  included) 

(4)  Second  Questionnaire  (not  included) 

(5)  Additional  Information  Requested  on  Pass  #1 

1.  The  responses  to  the  first  questionnaire  have  been  analyzed  and  the  re- 
sults are  returned  herewith.  In  general,  the  data  received  was  about  what 
we  expected.  That  does  not  mean  that  the  rnswers  are  the  same  as  we  re- 
ceived on  previous  experiments.  We  cannot,  as  yet,  tell  how  similar  or  dis- 
similar your  responses  are  to  the  previous  ones.  We  can,  however,  detect 
the  internal  consistency  we  have  come  to  expect  between  each  individuals  ans- 
wers, question  to  question,  and  between  separate  individuals  and  the  rest  of 
the  panel. 

2.  The  enclosures  provide  the  material  necessary  to  complete  your  second 
questionnaire.  A few  of  the  concepts  introduced  in  the  first  questionnaire 
proved  to  be  particularly  difficult.  As  a result  these  concepts  are  dis- 
cussed further  in  enclosure  (l).  Most  of  the  panelist's  rationale  state- 
ments centered  around  a few  points  and  these  have  also  been  summarized  in 
enclosure  (l)  together  with  some  particularly  thorough  rationale  statements 
from  some  of  the  participants.  The  rationale  has  been  summarized  across 
all  panels  so  you  are  seeing  comments  from  all  three  panels.  Enclosure  (2) 
is  your  own  first  questionnaire  returned  as  you  filled  it  out.  Where  your 
answers  to  a specific  question  were  statistically  different  from  the  rest 
of  your  panel,  those  answers  have  been  circled  in  red.  Pay  particular  at- 
tention to  these  questions.  If  you  feel  that  your  answer  is  more  valid 
than  that  of  the  rest  of  the  panel,  we  will  need  a statement  of  why  you  be- 
lieve so.  Don't  back  down;  if  you  think  you  are  right,  say  so! 

3*  Enclosure  (3)  shows  the  average  answer  given  to  each  question  by  your 
panel  members.  Remember  these  are  the  averages  of  your  panel  only.  En- 
closure (4)  is  a blank  questionnaire  to  be  filled  out  on  this  second  pass. 
Enclosure  (5)  is  the  additional  information  that  was  requested  by  individ- 
uals regardless  of  panel  membership.  It  may  or  may  not  be  of  use  to  you 
in  continuing  the  experiment. 

4«  There  is  no  particular  way  to  go  about  completing  this  second  step. 

Let  me  suggest  one  way  that  may  be  useful.  After  reading  through  the 
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whole  package  thoroughly,  take  it  apart  and  lay  enclosure  (2),  (3),  and  (4) 
side  by  side.  Evaluate  your  first  answer  in  light  of  the  panel  averages 
and  the  additional  information  you  now  have  and  then  record  your  new  re- 
sponse on  the  blank  questionnaire.  Remember  to  pay  particular  attention  to 
those  questions  we  have  marked  in  red  on  your  original  questionnaire. 


5.  You  need  return  only  your  answers  to  questions  1 and  2 on  the  last  page 
of  enclosure  (l)  and  your  completed  enclosure  (4)  in  the  self  addressed  en- 
velope. Save  the  remainder  of  this  package  for  use  in  the  next  pass. 


R.  K.  LEHTO 
SPECIAL  ASSISTANT  FOR 
ENLISTED  FORCE  ANALYSIS 
BUREAU  OF  NAVAL  PERSONNEL 
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ADDITIONAL  DISCUSSION  OF  EXPERIMENT 

General  Comment: 

Despite  a good  bit  of  griping  about  the  problems  of  dealing  with  the 
average  man,  surprisingly  high  percentage  of  you  (higher  than  usual)  were 
able  to  discuss  the  problem  and  provide  internally  consistent  answers  to 
the  questions.  The  average  man  concept  is  troublesome,  but  perhaps  if 
viewed  in  a different  way  it  may  be  a little  easier  to  deal  with.  The  av- 
erage progressor  is  not  any  one  single  man  who  progresses  from  0 to  30  years 
LOS.  Rather  he  is  a fictional  central  figure  representing  all  the  people 
in  his  grade  and  LOS  at  any  particular  time.  In  the  period  from  zero  to  a- 
bout  4 to  7 years,  depending  on  the  skill,  the  average  man  is  the  average 
of  both  those  who  stay  and  those  who  will  go  home.  In  the  years  from  4 to 
20  the  average  progressor  represents  those  who  "go  the  distance"  as  well 
as  those  who  drop  along  the  way.  He  also  represents  the  mix  of  the  fast, 
medium  and  slow  advancers.  Beyond  20  years,  the  average  progressor  is  the 
average  of  those  who  stayed  past  twenty.  You  must  evaluate  these  kind  of 
people  to  get  an  average,  not  only  among  themselves,  but  also  as  compared 
to  those  who  left  at  20. 


Please  examine  closely  your  answers  to  Question  C.  Remember,  if  you 
say  that  a man  reaches  50$  of  his  maximum  value  by  the  time  he  is  an  E4 
you  are  saying  that  half  of  all  this  guy  can  contribute  is  under  his  belt 
in  the  first  2 or  3 years.  It  will  take  him  17  to  28  years  to  get  the 
other  half  of  his  value.  The  answers  to  Question  C.  are  the  most  impor^- 
tant  part  of  this  first  phase  of  the  questionnaire.  Consider  them  care- 
fully. 


Panels  Rationale  Statements: 

Most  of  the  rationale  statements  dealt  with  the  concept  of  utility, 
the  average  man  (discussed  above),  or  the  reasons  for  the  answers  given 
about  the  average  progressor.  All  have  been  noted,  but  have  only  been 
quickly  summarized  here,  because,  in  general,  they  are  the  same  as  were 
discussed  in  the  original  briefing.  There  were,  however,  two  particular 
points  in  most  of  the  rationale  statements,  which  merit  special  attention. 


(l)  Most  people  discussed  their  rationale  for  the  decline  in  value 
at  a pay  grade  after  a certain  period  of  time.  In  general,  they  cited 
disillusionment,  lack  of  motivation,  lack  of  challenge,  or  just  "in  a rut." 
We  accept  this  decline  in  value  at  each  pay  grade,  but  the  important  ques- 
tion is  how  much  does  it  decline.  Pay  particular  attention  to  the  form 
of  the  question.  Where  we  ask  for  $ of  maximum  value  in  Question  B.,  we 
don't  mean  in  comparison  to  an  E9  necessarily  but  in  comparison  to  the  max- 
imum attained  at  that  pay  grade.  For  example,  if  you  said  that  an  E5  was 
at  his  peak  from  7 to  12  years,  we  want  to  know  in  percentage  terms,  how 
much  of  that  peak  value  does  he  have  left  if  he  goes  for  30  and  stays  at 
E5?  A similar  approach  should  be  taken  to  each  other  pay  grade. 


(2)  The  question  of  the  value  of  DPPO's  was  especially  difficult  as 
many  of  you  pointed  out  during  the  briefings.  We  must,  however,  determine 
a value  for  people  at  early  years  in  pay  grade  regardless  of  how  they  got 
there.  Perhaps  one  way  to  look  at  it  is  to  say:  A man  who  has  only  one 

year  of  service  at  a paygrade  (for  example  E5)  got  there  because  of  tech- 
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nical  ability  but,  as  many  people  pointed  out,  he  lacks  military  leader- 
ship skills  and  therefore  does  not  have  the  full  value  of  a man  who  grew 
up  in  the  system.  Therefore,  one  measure  of  a man’s  value  at  any  pay 
grade  in  the  first  year  of  service  is  the  percentage  of  that  pay  grade's 
value  which  can  be  attributed  as  technical  as  opposed  to  military  skills. 

I realize  this  won't  help  much  at  and  E9,  but  it  may  help  in  answering 
Question  D for  pay  grades  E7  and  below. 

Selected  Rationale  Statements: 

Three  rationale  statements  are  attached  here  verbatim  as  Tabs  A,  B, 
and  C.  They^were  chosen  as  representative;  there  were  many  others  as  good. 
Some  people  worked  nut  curves  and  even  completed  tables  for  us.  We  did  not 
include  these  rationale  statements,  but  will  deal  with  them  on  pass  three. 
For  those  of  you  who  did  draw  curves  or  filled  out  tables,  you  would  be 
surprised  at  how  close  your  answers  look  to  the  averages  of  the  panels  and 
to  the  results  of  the  previous  experiments. 

Additional  Questions  to  be  Answered: 

Only  one  additional  piece  of  information  will  be  addressed  in  this 
second  round.  In  future  rounds,  we  will  be  addressing  the  question  of  pay 
grade  overlap,  that  is  the  extent  to  which  the  value  curves  overlap  from 
pay  grade  to  pay  grade.  In  order  to  help  us  write  the  appropriate  ques- 
tions, we  would  like  your  views.  On  the  bottom  or  back  of  this  sheet 
please  answer  the  following  questions.  Remember,  we  are  primarily  inter- 
ested in  the  LOS  dimension. 

1.  Under  what  circumstances  do  you  believe  that  the  average  E5  is 
worth  more  than  the  average  E6? 

2.  Under  what  circumstances  is  the  average  E5  worth  less  than  the 
average  E4? 


A- 1 2 


Enclosure  ( 1 ) 


PANELIST’S  RESPONSE  I 


ASSUMPTIONS /RATIONALE 

A.  I have  had  little  direct  experience  with  petty  officers  with  more  than 
20  years  service;  my  answers  are  based  on  this  limited  experience  and,  I 
feel,  are  stretched  towards  the  30  mark  as  a result  of  the  format  of  the 
questionnaire. 

B.  The  answers  to  questions  A1  and  A2  depend  on  a large  degree  to  the  bil- 
let assignment  and  the  demands  and  challenges  placed  on  that  billet.  In 
many  billets  there  is  a level  of  proficiency  required  where  more  experienc- 
ed and  more  highly  trained  personnel  do  not  necessarily  produce  improved 
performance.  I feel  that  there  is  a bum  out  effect  whereby  a top  young 
performer  can  have  his  performance  decline  when  exposed  to  long  term  rigors 
of  a demanding  position.  When  removed  from  this  position  (perhaps  one 
where  he  has  served  before;  e.g.  SSN  Sonarmen,  MTs  and  FTs,  and  some  nuc- 
lear billets),  he  can  reach  new  performance  levels  through  the  challenge 

of  a different  assignment. 

C.  Question  A4  is  rating  dependent.  This  percentage  is  achieved  earlier 
in  less  skilled  ratings. 

D.  Question  D was  answered  on  judgment;  I have  no  experience  in  this  mat- 
ter. This  "value"  would  be  largely  dependent  on  the  person  recruited  and 
the  recruiter's  adherence  to  well-defined  standards  which  have  been  pre- 
viously established. 
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PANELIST'S  RESPONSE  II 


mmmm 


Basic  rationale 

for  my  answers 

is  contained 

in  the  chart  below; 

A 

Minimum 
Time  In 
Previous 
Grade  For 
Eligibility 

B 

Minimum 
Longevity 
To  Advance 
To  Grade 

C 

ffy  Guess 
Average 
Longevity 
To  Advance 
To  Grade 

D 

My  Guess 
Average 
Eligibility 
To  Advance 
To  Next  Grade 

E 

Arbitrary 
Decline  Point 
Column  D+2  Yrs 

E9 

2 years 

12.5  years 

18  years 

24 

E8 

3 years 

10.5  years 

16  years 

18  years 

20 

m 

3 years 

7.5  years 

10  years 

16  years 

18 

E6 

2 years 

4.5  years 

6 years 

10  years 

12 

E5 

1 years 

2.5  years 

3.5  years 

6 years 

8 

| , E4 

6 months 

1.5  years 

2.0  years 

3.5  years 

5.5 

E3 

8 months 

1.0  years 

1.2  years 

2.0  years 

3.0* 

E2 

4 months 

4 months 

4 months 

1.2  years 

2.0* 

El 


* Exceptions  to  Dt-2  years  if  no  advancement  by  years  shown,  demotivation,  "short" 
attitude  and  poor  performance  follows. 

Rationale  by  Questions; 

A. l.  Maximum  value  to  the  service  comes  at  (E-9/C) 

2.  Arbitrary  decline  point  is  (E-9/E) 

3»  Pure  guess 

4.  By  the  end  of  4 years  average  progressor  should  be  an  E-5,  where 
I value  at  60$. 

5»  50$  value  should  come  just  before  E-5,  i.e.  3*5  years  (E-4/D)  or 

E-  5/G-something ) 

B. l.  Maximum  year  equates  to  column  D,  when  a man  should  be  ready  for  ad- 

vancement. If  my  estimate  is  wrong,  crank  in  actual  figures  for 
column  D and  that  is  my  answer. 

Maintain  year  equates  to  column  E.  After  a man  goes  2 years  past  his 
average  eligibility  for  advancement  I believe  he  cannot  maintain  max- 
imum utility  due  to  new  self  image,  which  has  to  be  adjusted  downward 
to  match  level  attained. 

Value  at  30th  year  is  a pure  guess.  I don't  think  those  figures  are 
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worth  looking  at.  We  can  expect  decline . We  are  already  guessing 
where  it  takes  place.  To  superimpose  a guess  on  a guess  is  a wasted 
effort. 

C.  Jump  at  E-4  and  E-5  is  due  to  experience  as  a worker.  E-6  added  value 
is  in  the  supervisory  skills  primarily.  Additional  value  of  E-7  is 
due  to  leadership  development  and  his  acceptance  as  "The  Boss"  in  a 
shop.  E-8  and  E-9  gains  are  small  in  this  area.  I still  think  this  a 
helluva  a way  to  try  to  rationalize  our  manpower  structure.  There  are 
better  ways  to  come  up  with  these  answers  than  a figure  that  represents 
an  agreed-upon  guess  or  an  unidentifiable  quantity:  "The  worth  of  an 

average  man". 

But  I’ll  play  the  next  round. 
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Comments  and  suggestions: 

A.  The  assumption  that  the  average  Navyman  completes  a thirty  year  ca- 
reer as  an  E9  is  not  valid.  I feel  the  average  man  will  progress  only 

to  E6  or  E7.  The  average  E8  and  E9  are  or  should  be  exceptional  individ- 
uals. 

1.  Maximum  value  of  an  E9  does  not  occur  until  about  25  years. 

2.  Maximum  value  of  many  individuals  continues  beyond  30  years. 

A..  Major  training  was  completed  during  the  first  enlistment.  Addi- 
tional training  and  experience  will  increase  value,  but  diminishing  re- 
turns set  in  beyond  this  point. 

B.  Value  of  an  individual  as  related  to  cost  remains  fairly  constant,  but 
value  as  a source  of  individuals  who  can  progress  to  higher  pay  grades  de- 
creases linearly. 

C.  The  value  of  individuals,  by  pay  grade,  depends  upon  many  factors: 

i Time  required  to  make  a pay  grade,  capabilities  of  individuals  in  a given 

pay  grade,  training  required,  money  invested,  usefulness,  requirements, 
etc.  While  individuals  in  higher  pay  grades  are  usually  assumed  to  be  capa- 
ble of  performing  any  task,  the  fact  is  that  certain  tasks  require  special- 
ists in  lower  grades.  Value  based  on  cost  is  low  for  an  individual  assign- 
ed to  any  job  which  could  be  performed  by  an  individual  in  a lower  pay  grade. 
To  maximize  value,  individuals  must  progress  as  high  as  their  capabilities 
permit  and  they  must  be  utilized  to  the  maximum  extent  of  those  capabili- 
ties. The  most  qualified  individual  in  paygrade  E9  has  no  value  if  he  is 
not  utilized. 

D.  The  overall  value  of  these  individuals  is  almost  equal  to  that  of  their 
military  contemporaries.  The  major  drawback  is  the  time  required  to  gear 

[ them  to  the  military  environment.  Also,  much  depends  on  the  specific  task 

| for  which  they  were  recruited;  some  billets  can  be  filled  only  by  well  qual- 

ified men  with  a comprehensive  military  background  while  others  require  al- 
most no  military  background  at  all. 
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Imiosure  ( I ) 
Tab  C 


Enclosures  2,  3,  and  4 to 
second  Delphi  Questionnaire  not  reproduced 
in  this  appendix. 
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Additional  Information  Requested  on  Pass  #1 

1.  Average  time  in  grade  when  advanced,  based  on  pay  entry 
base  date  (PEBD) . 


E2 

E3 

E4 

E5 

E6 

E7 

E8 

E9 

.3 

.9 

1.6 

1.7 

3.6 

5.5 

4.5 

3.0 

2. 

Average  i 

time  in 

the  Navy  when  advanced. 

based 

on  PEBD. 

E2 

E3 

E4 

E5 

E6 

E7 

E8 

E9 

.4 

1.1 

2.3 

3.7 

9.0 

14.8 

16.9 

19.6 

3. 

Paygrade 

(PG)  vs 

length 

. of 

service 

(LOS) 

of  the  average 

enlisted 

man 

based  on  June 

1972 

inventory. 

El 

E2 

E3 

E4 

E5 

E6 

E7 

E8 

E9 

.6 

1.1 

2.2 

3.4 

6.6 

13.1 

17.4 

19.7 

22.7 

Retirements  (Disability  not  Included) 

Length  of  service  (LOS)  and  paygrade  (PG)  of  retirements  from  USN 


during  first  six  months  of  1973. 


LOS 

E9 

E8 

E7 

E6 

Undeter- 

mined 

2 

4 

24 

65 

17 

1 

3 

21 

75 

18 

19 

43 

239 

294 

19 

48 

94 

559 

495 

20 

24 

77 

288 

189 

21 

32 

61 

132 

62 

22 

22 

28 

57 

14 

23 

6 

9 

19 

7 

24 

4 

13 

21 

8 

25 

9 

11 

18 

7 

26 

7 

8 

25 

2 

27 

8 

11 

13 

10 

28 

9 

4 

1 

29 

2 

7 

5 

30  or  more 

3 

10 

Total 

Per  PG 

182 

373 

1430 

1244 

% Per  PG 

5.15 

10.55 

40.44 

35.18 

L. 


Total 

% Per 

E5 

E£ 

E3 

Per  LOS 

LOS 

16 

1 

112 

3.17 

8 

108 

3.05 

27 

1 

623 

17.62 

148 

22 

3 

1369 

38.72 

38 

8 

1 

625 

17.67 

17 

304 

8.60 

3 

124 

3.51 

41 

1.16 

2 

1 

49 

1.38 

4 

49 

1.38 

1 

43 

1.22 

3 

1 

46 

1.30 

14 

.40 

2 

16 

.45 

13 

.37 

269 

34 

4 

3536 

.61 

.96 

.11 

100.00 

Enclosure  (5) 


DEPARTMENT  OF  THE  NAVY 

BUREAU  OF  NAVAL  PERSONNEL 
WASHINGTON.  D.C.  20370 


r 

F 

i' 


i 


IN  PUM.Y  RUCK  TO 

Pers-2x/mmo 
Ser  1010/73 
Nov  12  1973 


MEMORANDUM  FOR  PARTICIPANTS 


IN  DELPHI  EXPERIMENT 


Subj  : 


DELPHI  Determination  of 


Utility;  third  questionnaire 


Enel:  (1)  Additional  Discussion  of  Experiment 

(2)  Graph  of  Responses  from  Second  Questionnaire 

(3)  Rational  Statements  from  First  and  Second  Questionnaires 

1.  We  have  analyzed  the  results  received  from  the  second  questionnaire. 

In  general,  the  experiment  is  continuing  along  about  as  we  expected.  This 
does  not  mean  that  everybody  is  marching  in  step.  On  the  contrary,  we  have 
some  people  who  are  strongly  opposed  to  the  view  of  the  majority.  Some  ol 
their  views  are  represented  in  the  rationale  statements  in  enclosure  (3). 
When  I say  the  experiment  is  going  well,  1 mean  that  a consensus  is  emerg- 
ing and  that  each  of  the  answers  has  roughly  a normal  distribution  about 
some  ..lean  value.  For  those  who  complained  that  they  were  being  forced  to 
change  their  answers  to  agree  with  the  majority,  nothing  could  be  further 
from  the  truth.  Of  course,  we  are  trying  to  obtain  relative  agreement; 
that's  what  this  experiment  is  all  about.  We  are  trying  to  find  agreement 
on  values,  where  agreement  is  possible.  If,  however,  you  feel  you  cannot 
agree  with  the  majority,  feel  free  to  say  so  and  do  so.  Because  ultimately, 
we  will  use  a distribution  of  answers,  every  person's  views  will  bear 
weight  in  the  final  answer. 


2.  We  are  a little  concerned  with  the  number  of  people  who  are  not  send- 
ing back  their  questionnaires.  If  we  continue  to  lose  people  on  each  pass, 
we  will  not  have  enough  remaining  answers  to  do  the  cross  comparisons  which 
all  panels  indicated  were  of  the  most  interest  to  them.  As  a partial  stop 
gap  we  are  returning  this  questionnaire  to  all  those  who  answered  the  first 
one  even  if  they  did  not  respond  to  the  second.  The  way  this  questionnaire 
is  structured,  everyone  will  have  enough  information  to  respond  even  if  he 
did  not  complete  the  second  questionnaire. 

3.  Enclosure  (1)  includes  some  additional  discussion  concerning  the  experi- 
ment and  the  instructions  for  filling  out  this  questionnaire.  Enclosure  (2) 
contains  a graph  which  summarizes  the  two  previous  questionnaires,  and 

also  serves  as  the  answer  sheet  for  this  one.  Enclosure  (3)  includes 
excerpted  rationale  statements  from  botli  the  first  and  second  questionnaires. 
The  selection  of  statements  was  made  by  picking  examples  of  statements 
which  highlighted  the  important  points  of  discussion. 
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Subj : DELPHI  Determination  of  Utility;  third  questionnaire 

4.  Please  return  responses  as  rapidly  as  possible.  We  hope  to  use  the 
results  of  this  experiment,  when  completed,  to  meet  a deadline  of  January 
1974  as  levied  by  the  Chief  of  Naval  Personnel. 


R.  K.  LEHTO 

SPECIAL  ASSISTANT  FOR 

ENLISTED  FORCE  ANALYSIS 
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ADDITIONAL  DISCUSSION  OF  EXPERIMENT 


Central  Comment: 

This  particular  questionnaire  will  deal  primarily  with  the  utility  of 
the  average  man  at  each  paygrade.  This  does  not  mean  we  are  abandoning  the 
average  progressor  concept.  We  will  return  to  it  on  the  fourth  and  last 
iteration. 


Enclosure  (2)  is  a graph  of  the  average  responses  from  your  panel 
translated  into  graphic  form.  The  information  from  Section  B which  asked 
for  the  year  reaching  peak  value,  the  year  to  which  that  value  was  main- 
tained, and  the  value  of  the  30th  year  was  used  to  set  the  shape  of  each 
paygrade  curve  at  the  top  and  the  right  hand  side.  The  information  from 
Section  D,  on  DPPO's,  was  used  to  set  the  shape  of  the  left  hand  side  of 
each  curve.  The  height  of  each  paygrade  curve  was  determined  by  your 
answers  to  Section  C where  we  asked  specific  paygrade  values  by  percentage. 
We  have  abandoned  the  data  on  paygrades  El  and  E2  because  it  was  obvious 
from  the  first  two  questionnaires  that  no  meaningful  consensus  was  develop- 
ing. 


To  repeat  then,  enclosure  (2)  is  a graph  which  represents  the  sum  of 
most  of  the  answers  you  provided  in  the  first  two  questionnaires.  The  pur- 
pose of  this  questionnaire  is  to  permit  you  to  evaluate  the  results  of  your 
work  when  presented  in  a different  way.  In  other  words,  here  is  your  chance 
to  disagree  with  the  way  things  are  turning  out.  Remember,  each  line  of 
the  graph,  labeled  E3,  E4,  etc.  represents  the  utility  of  an  average  man  at 
that  paygrade  at  any  given  point  in  time  in  a thirty  year  career. 

Graphic  Representation  of  Information  Obtained: 

Most  of  your  responses  to  this  questionnaire  can  be  indicated  by  writ- 
ing or  drawing  right  on  the  graph  itself  in  enclosure  (2).  If  you  want  to 
make  further  comment  use  the  back  of  enclosure  (2)  or  attach  extra  sheets. 

If  the  curves  in  enclosure  (2)  do  not  agree  with  your  views,  redraw  the 
curves  to  suit  yourself.  It  is  unnecessary  to  use  straight  lines.  If 
curved  lines  express  your  views,  use  them.  We  will  extract  the  data  from 
the  graphs  based  on  where  you  redraw  them.  In  those  portions  of  the  curves 
where  you  did  not  redraw  the  curves  we  will  assume  you  agree  with  the 
results  in  that  portion. 

I believe  that  the  three  key  grades  in  this  experiment  are  E3,  E5,  and 
E7.  If  you  want  to  simplify  your  task  and  concentrate  on  those  three  pay- 
grades,  check  the  box  "Three  Paygrade  Option"  on  the  graph  and  confine  your 
answers  to  paygrades  E3,  E5,  and  E7. 


First  examine  the  flat  tops  of  the  curves.  These  bars  or  lines  on  the 
top  represent  the  period  of  time  that  the  average  man  is  at  peak  effective- 
ness in  that  paygrade.  Even  more  importantly  the  height  of  the  line  above 


A-22 


ENCLOSURE  (1) 


zero  on  the  graph  represents  that  paygrade's  utility  in  reference  to  all 
other  paygrades.  Consider  the  following  questions  and  then  redraw  the  graph 
as  you  see  fit. 

Does  each  of  the  flat,  top  lines  start  and  end  at  about  the  right 
year?  Does  the  length  of  the  flat,  top  line  seem  to  reflect  the 
length  of  time  a man  is  at  maximum  effectiveness?  Are  some  of 
the  lines  too  close  together  or  too  far  apart? 

Consider  now  the  right  hand  side  of  the  curves  (the  decline  after  passing 
peak  utility).  Again  redraw  the  curves  if  you  wish  based  on  your  views  and 
the  following  considerations. 

Does  the  average  man  lose  effectiveness  faster  than  this  or  slower? 
Notice  the  amount  of  overlap.  An  old  E6  can  be  worth  less  than  a 
young  E5  as  currently  expressed  by  the  graphs.  Do  you  think  this 
is  so?  Should  the  overlap  be  more  or  less? 

Consider  now  the  left  hand  side  of  the  curves  (the  rise  from  zero  years 
to  peak  utility).  Again,  redraw  the  lines  if  you  wish. 

Should  these  very  young  (in  terms  of  military  experience)  people 
be  worth  more  or  less  than  shown?  Is  there  too  much  or  too  little 
spacing  between  the  paygrades?  Note  the  overlap  on  this  side.  In 
some  cases  a DPPO  E9  is  worth  less  than  an  E5  who  "grew  up  with  the 
system."  Is  this  reasonable? 

Additional  Information  from  this  Questionnaire: 

After  completing  this  step  you  should  have  a marked  up  graph  which 
reflects  your  adjustments  to  information  we  currently  have.  If  everything 
is  tracking  so  far  the  curves  can  be  interpreted  as  the  change  in  utility 
of  the  average  man  by  paygrade.  If  this  is  the  case,  then  it  follows,  by 
definition,  that  the  average  man,  who  has  just  been  advanced  to  grade,  lies 
somewhere  on  the  line  for  that  grade.  This  is  the  only  new  piece  of  in- 
formation we  require  on  this  pass.  Place  an  X on  each  line  (E3,  E5,  E7  if 
you  selected  that  option)  at  the  point  you  think  represents  the  utility  of 
the  average  man  advancing  to  that  paygrade.  For  example,  considering  the 
E5's,  where  on  the  E5  line  is  the  man  who  just  got  advanced  to  E5?  Repeat 
this  for  each  other  paygrade  you  are  considering. 

Comments  and  Suggestions: 

Finally,  make  any  comments  you  wish,  put  your  name  and  address  on  the 
graph  and  send  just  the  graph  and  any  attached  comment  sheets  back  in  the 
enclosed  envelope. 
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RATIONAL  STATEMENTS  FROM  FIK3-  AND  SECOND  QUESTIONNAIRES 


The  following  is  a summary  of  comments  from  all  panels.  The  comments 
deal  mainly  with  the  shape  of  the  paygrade  curves  and  may  or  may  not  affect 
your  answers.  Before  you  are  swayed  by  these  comments,  remember,  these  are 
comments  which  support  outliers.  The  curves  represent  the  panel  average 
and  therefore,  they  represent  the  "implied  comments"  from  the  majority. 

1.  The  DPPO  question  was  used  to  determine  the  initial  value  of  the 
paygrades.  Remember,  the  DPPO  is  recruited  based  only  on  his  technical 
skills  gained  from  non-Navy  sources,  but  he  will  be  required  to  fill  a bil- 
let at  his  paygrade  which  includes  all  functions  of  that  paygrade.  Although 
no  large  statistical  differences  were  noted,  the  following  arguments  were 
given  in  support  of  answers: 

a.  One  argument  says  the  lower  paygrade  values  should  be  higher 
because  it  does  not  take  long  to  teach  a civilian  to  swab  or  polish  bright 
work.  Also  a high  school  graduate  with  some  trade  school  training  could 
make  an  effective  E4  or  E5  if  he  was  not  required  to  perform  military 
duties.  (The  point  is — he  will  be  required  to  perform  military  duties 
and  this  will  reflect  in  his  value  of  the  Navy.) 

b.  Some  argue  that  the  senior  petty  officer  has  to  have  skills 
which  can  only  be  obtained  within  the  Navy,  i.e.,  Naval  leadership,  career 
counselor,  in  depth  background  information,  military  bearing  and  familiarity 
with  the  "Navy  way."  Others  argue  that  E8  and  E9  are  basically  adminis- 
trative and  middle  management  positions  and  these  traits  can  be  developed 
outside  the  Navy  with  little  or  no  difference  in  value. 

2.  For  the  flat  portion  of  the  paygrade  curves,  the  maximum  value  was 
determined  from  question  ' C'  in  which  a paygrade  value  relation  was  re- 
quested; and  the  length  and  LOS  placement  were  determined  from  question  'B’ 
using  the  "Maximum  Year,"  the  year  maximum  value  was  attained,  as  the  begin 
point  and  the  "Maintain  Year,"  the  year  the  man  will  maintain  maximum  value, 
as  the  point.  Support  for  maximum  value  follow: 

a.  A panelist,  whose  paygrade  value  for  E6  and  below  averaged  13 
points  lower  than  the  panel's  average,  argues  that  the  value  should  be 
lower  for  E4  and  below  and  higher  at  E7  and  above.  He  says  the  greatest 
value  step  should  come  between  F,5  and  E6  and  most  change  in  value  (perhaps 
70%  or  more)  occurs  between  E4  and  E7.  Other  panelists  place  E4  value  as 
high  as  70%  leaving  relatively  little  value  differential  for  the  higher 
paygrades. 


b.  One  panelist  argues  that  a person  reaches  50%  of  his  max  value 
before  seven  years  LOS  but  that  an  E7  maximum  value  is  only  70%.  He  argues 
that  since  a distinction  is  made  for  E8  and  E9  billet  writing  then  there 
should  be  a greater  value  differential  for  these  paygrades.  Another  argu- 
ment says  there  is  no  value  differential  between  the  average  E7  and  E9. 
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c.  One  panelist  considers  that  E7  is  the  most  valuable  pay- 
grade  and  not  E9.  He  argues  that  the  E8's  and  E9's  are  generally  in 
administrator  or  instructor  billets  ashore  and  that  E8's  and  E9's  at 
sea  in  non-technical  billets,  i.e.,  PMS  coordinator,  CMAA's,  career 
counselor,  etc.  They  are  divorced  from  the  rating  in  which  their  value 
was  obtained.  He  says  that  the  E7  has  the  ideal  balance  of  technical 
ability  and  administrative  knowledge  which  makes  E7  the  most  valuable 
paygrade.  (Should  the  average  E7  be  paid  more  than  the  average  E8  or  E9?) 

3.  There  is  some  disagreement  with  the  value  a man  has  if  he  remains 
at  a lower  paygrade  till  his  30th  year.  The  arguments  follow: 

a.  The  man  who  stays  in  the  Navy  at  a low  paygrade  for  30  years 
is  the  "professional  seaman"  and  is  satisfied  with  his  position,  there- 
fore, maintains  a very  high,  if  not  100  percent  of  his  maximum  value. 

b.  The  lower  paygrade  values  drop  too  low.  The  skills  required 
for  the  lower  paygrades  are  not  significantly  degraded  over  time  thus  value 
remains  high.  Even  accounting  for  lost  motivation  and  limited  ability  the 
value  of  a nan  in  the  lower  paygrades,  E3-E5,  would  not  fall  lower  than 
50%  of  his  maximum  value. 

c.  One  panelist  feels  that  the  lower  paygrades  E3-E5  are  more 
of  a problem  than  they  are  worth.  In  fact,  at  the  20th  year  and  before 
they  present  similar  problems  e.g.,  they  feel  that  they  rate  certain 
privileges  accorded  only  to  their  seniors  in  grade  solely  due  to  their  time 
in  the  service;  they  generally  have  severe  monetary  problems;  their  self 
image  is  poor  which  is  usually  accompanied  by  other  psychological  hangups; 
to  mention  only  a few  problems.  An  E3  would  fall  into  this  same  pit  by 
the  12th  year. 

d.  The  E7  who  remains  more  than  a few  years  past  the  time  he 
could  retire  drops  to  30%  of  his  maximum  value.  The  drop  occurs  from  the 
25th  to  30th  year  and  is  caused  by  the  fact  that  since  he  does  not  advance 
he  is  just  staying  around  for  security;  he  does  not  keep  up  with  his  rate 
and  he  resists  new  programs.  This  is  a much  larger  drop  than  any  other 
paygrade,  including  E8  or  E9. 

4.  Tabs  A,  B,  and  C are  Tables  and  graphs  submitted  by  individual  panelists 
They  are  included  as  additional  information  that  may  introduce  a different 
way  of  considering  utility,  and  assist  in  answering  this  questionnaire. 

For  item  C,  I have  included  two  columns  as  shown  in  Figure  1.  Column 
A represents  as  originally  stated,  the  maximum  value  of  a man  at  a particula 
paygrade  compared  to  an  E9  e.g.,  approximately  6 E2's  would  have  the  same 
value  as  an  E9,  or  2 E4's  at  their  maximum  value  would  be  worth  one  E9. 
Column  B has  the  percentage  of  maximum  value  that  an  individual  contributes 
as  he  rises  up  througli  tiie  ranks  to  E9,  considering  E9  as  100%  e.g.,  at 
tile  E6  level  a man  has  contributed  50%  of  his  worth. 
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TAB  B to  Knot  3 


Rationale  Statements  from  First  and  Second  Questionnai re 


GROUP  UTILITY  CURVE 


Length  of  Service 


1.  This  chart  is  more  reflective  of  the  average  individual  in  each 

year  of  service  rather  than  the  individual's  progression  thru 
service  experience.  (See  next  chart  for  individual  progression 
using  the  study's  criteria.  ) 

2.  Change  in  slope  @ 4-5  yrs  reflects  separation  and  short  timers-- 

perhaps  "dropoff"  should  be  even  more  exaggerated. 

3.  "Drop-off”  18-20  yrs  reflects  retirees  and  short  timers. 

4.  Max  "utils"  occurs  « 17  yrs,  but  approaches  100  after  20  and 

thru  30+  yrs.  Effectively,  the  individual  maintains  a nearly 
constant  "util"  from  17  yrs  until  30+  yrs. 


INDIVIDUAL  UTILITY  CURVE 


Length  of  Service 


lindosure  (3) 
Tab  B Page  1 


4 


UTILITY  RY  PAYGRADE 


Paygrade 

Notes 

1.  Paygrade  is  at.  promotion  --it  does  not  accurately  reflect 
time  in  grade. 

2.  This  chart  is  also  group  oriented,  and  not  directly  oriented 
to  the  E1-E9,  30  yr.  individual. 
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MEMORANDUM  FOR  PARTICIPANTS  IN  DELPHI  EIPERIMENT 

Subj:  DELPHI  Determination  of  Utility*  clarification  on 

Third  Questionnaire 

Ref:  (a)  Pera-2x  memo  Pers-2x/ramo  ser  1010/73  of  12  NOV  73 

Enel:  (1)  Graph  of  Responses  from  Second  Experiment 

1.  Early  responses  received  from  the  third  questionnaire 
indicate  we  may  have  a problem.  The  only  "newu  information 
required  from  this  pass,  and  therefore  the  most  important, 
has  been  overlooked  by  some  participants.  Without  this  addi- 
tional information,  the  third  questionnaire  will  be  much 
reduced  in  value;  therefore,  the  applicable  portion  of  the 
third  questionnaire  is  being  repeated  here: 

Additional  Information  from  This  Questionnaire: 

"After  completing  this  step  you  should  have  a marked  up 
graph  which  reflects  your  adjustments  to  information  we 
currently  have.  If  every  tiling  ie  tracking  so  far  the  curves 
can  be  interpreted  as  the  change  in  utility  of  the  average 
man  by  paygrade.  If  this  is  the  case,  then  it  follows,  by 
definition,  that  the  average  man,  who  has  just  been  advanced 
to  grade,  lies  somewhere  on  the  line  for  that  grade.  This  is 
the  only  new  piece  of  information  we  require  on  this  pass. 
Place  an  X on  each  line  (E3,  E5,  E7  if  you  selected  that 
option)  at  the  point  you  think  represents  the  utility  of  the 
average  man  advancing  to  that  paygrade.  For  example,  consi- 
dering the  ES's,  where  on  the  E5  line  is  the  man  who  just  got 
advanced  to  K5?  Repeat  this  for  each  other  paygrade  you  are 
considering. " 

In  addition,  another  copy  of  the  graph  from  reference  (a)  is 
included  as  enclosure  (1) . 

2.  If  you  have  already  mailed  your  response  to  the  third 
questionnaire  with  the  "additional  information”  from  the 
paragraph  quoted  above  please  disregard  this  memo.  If  not, 
wo  urge  you  to  complete  the  third  questionnaire  with  all 
information  and  return  prompt!/  so  as  to  avoid  the  holiday 
mailing  crunch.  As  stated  before,  we  hope  to  use  this  to 
meet  a January  1974  deadline.  If  you  have  already  returned 
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iitM'h  1 nation  of  btiiltyj  clarification  on 

Tail  a juo»tiomi*lrn 

your  questionnaire  anu  iiJLrf  not  provide  tb©  information 
requested  above,  pleaao'^fnTiTcatu  your  responses  on  enclosure 
(1)  and  return. 


H.  K.  LEU  TO 
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MEMORANDUM  FOR  PARTICIPANTS  IN  DELPHI  EXPERIMENT 
Subj:  DELPHI  Determination  of  Utility;  last  questionnaire 

Enel:  (1)  Graph  of  Results  from  Third  Questionnaire 

(2)  Smoothed  Graph  of  Results 

(3)  Additional  Information  Required 

1.  This  questionnaire  completes  the  DELPHI  experiment  we  began  some 
time  ago.  Those  of  you  who  are  still  with  us  deserve  our  thanks  for  your 
time  and  consideration.  In  addition,  I think  you  can  take  a good  deal  of 
self  satisfaction  in  having  completed  (after  this  questionnaire)  a difficult 
series,  requiring  thought  and  time,  and  in  having  made  a very  significant 
contribution  to  the  long  range  planning  capabilities  of  the  Chief  of  Naval 
Personnel. 

2.  Reviewing  briefly  the  results  obtained  so  far  in  the  experiment  we  find 
that  the  present  panels  have  reconfirmed,  in  a more  refined  and  useful 
manner,  what  we  already  suspected  about  the  concept  of  the  average  pro- 
gressor.  In  addition,  you  have  made  contributions  to  what  we  believed 
about  the  relative  value  levels  of  the  average  man  at  each  paygrade.  Most 
significantly,  you  have  developed  and  described  for  us  the  utility  of  men 
by  paygrade  as  they  progress  through  their  career.  This  factor  we  knew 
little  about  before,  and  in  this  area  my  own  personal  biases  were  wrong. 

As  a result  of  your  efforts  we  have  been  able  to  correct  some  previous 
assumptions  which  could  have  caused  us  considerable  difficulties  in  the 
future. 

3.  Enclosure  (1)  is  the  graph  which  you  saw  on  the  last  pass,  corrected 
for  the  changes  which  people  made  on  the  last  pass.  As  nearly  as  we  can 
tell  this  represents  the  best  estimate  we  will  be  able  to  obtain  from  this 
experiment,  of  the  variation  of  utility  by  paygrade  and  LOS.  The  graph, 

as  you  remember  represents  only  the  plotting  of  4 points  for  each  paygrade. 
All  authorities  agree  that  utility  growth  over  time  does  not  change  linearly 
but  rather  changes  continually  in  smooth  curved  forms.  Some  of  you 
recognized  this  when  you  corrected  your  graphs  on  the  last  pass  and  not 
only  adjusted  points  but  also  hand  smoothed  the  line's.  What  we  have  done 
is  taken  the  four  points  you  have  given  us  and  fitted,  by  computer,  a smooth 
curve  which  describes  these  points  in  a continuous  curve  instead  of  a series 
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of  straight  lines.  The  result  of  this  curve  fitting  is  shown  in  enclosure  (2). 
The  picture  in  enclosure  (2)  is  the  result  of  the  weighted  average  of  all  panels 
combined  and  is  provided  here  to  show  you  the  final  results. 


4.  Enclosure  (3)  is  the  material  required  on  this  pass  and  requires  com- 
parisons of  the  average  man  with  men  in  specific  ratings.  It  is  the  final, 
and  one  of  the  more  important  pieces  of  information,  we  need  to  complete 
the  experiment.  Give  it  your  close  attention.  This  is  your  chance  to  ex- 
press the  point  most  of  the  people  raised  in  briefings.  "How  do  we  handle 
the  differences  between  ratings?" 

5.  The  final  results  of  the  experiment  will  be  available  about  a month  from 
now,  depending  on  how  fast  the  questionnaires  come  in.  If  enough  of  you 
are  interested,  we  will  come  down  to  Norfolk,  probably  late  in  April,  to 
brief  you  on  the  results  of  your  efforts,  how  you  compared  with  previous 
experiments,  and  most  importantly,  how  your  work  will  be  used  in  the 
future.  The  briefing  would  probably  be  conducted  at  one  central  location, 
probably  NOB  Norfolk,  rather  than  the  separate  briefings  previously  con- 
ducted. If  you  are  interested  in  such  a briefing,  there  is  a place  on  the 
answer  sheet  to  so  indicate. 

6.  Again,  thanks  for  your  time,  efforts,  and  cooperation.  You  have  made 
a contribution  far  greater  than  many  of  you  realize. 

R.  K.  LEHTO 

SPECIAL  ASSISTANT  FOR 

ENLISTED  FORCE  ANALYSIS 
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O HA  PI  1 OK  RESULTS  FROM  THIRD  QUESTIONNAIRE 

Thr-'e  ) 'nyprr;u|c  Opinr 


ADDITIONAL  INFORMATION  REQUIRED 


1.  There  are  two  graphs  attached,  Figure  I and  2,  one  of  which  you  will 
need  for  your  answer.  Figure  I is  the  same  as  enclosure  (1)  which  is  the 
same  graph  you  saw  last  pass  modified  by  the  changes  your  panel  made  on 
that  pass.  It  represents  the  utility  of  the  average  man  at  each  paygrade. 

Figure  2 represents  the  average  Navy  progressor  as  he  moves  through  his 
career  up  to  E9.  Some  of  you  probably  still  disagree  with  this  portrayal 
of  the  concept,  but  for  the  purposes  of  this  question,  please  assume  it  is 
correct.  We  are  interested,  in  this  questionnaire,  in  evaluating  the  differences 
between  the  average  Navy  man  and  the  average  man  in  a specific  rating  (or 

group  of  ratings).  In  order  to  do  this  we  must  start  from  some  common 
baseline. 

2.  The  table  attached  is  the  DOD  grouping  structure  which  is  generally  used 
to  aggregate  ratings  into  categories  based  on  the  kind  of  work  they  do. 

Again,  you  may  not  completely  agree  with  each  group,  but  in  general,  it  is 

a useful  grouping.  We  are  using  DOD  groupings  as  well  as  ratings,  because, 
if  we  use  ratings  only,  the  data  on  each  rating  will  be  so  small  we  will  not 
be  able  to  get  a good  statistical  analysis. 

3.  Instructions : From  the  DOD  grouping  table,  select  the  rating  group  you 
are  most  familiar  with  from  experience  and  for  which  you  feel  most  competent 
to  make  judgments  about  the  value  of  people  working  in  that  group.  Mark  the 
selected  group  with  an  X.  Within  the  group,  select  those  ratings  with  which 
you  are  most  familiar.  Circle  the  selected  ratings.  Choose  only  one  rating 
group  and  do  not  attempt  to  make  any  comparisons  between  rating  groups  except 
to  identify  the  group  with  which  you  are  most  knowledgeable.  We  assume  that 
your  answers  to  this  questionnaire  are  based  on  your  knowledge  of  the  ratings 
selected.  Now  choose  either  figure  1 or  2,  whichever  concept  you  prefer  to 
work  with  and  draw  in  the  curve  or  curves  representing  your  estimate  of 
utility  over  time  of  the  average  man  in  the  ratings  or  rating  group  you 

have  selected.  This  will  show  how  the  average  man  in  a specific  rating  or 
group  of  ratings  is  different  from  the  Navy-wide  average  man.  If  you  prefer 
working  with  the  paygrade  curves  in  figure  1 you  may  simplify  your  task  and 
concentrate  on  only  paygraderE3,  E5  and  E7.  Redraw  any  paygrade  curve  where 
you  feel  the  average  man  in  the  specific  rating  or  group  of  ratings  differs 
from  the  Navy-wide  average  man.  Remember,  the  graphs  given  you  show:  in 
figure  1,  the  value  overtime  of  the  average  Navy  man  at  each  paygrade  and  in 
figure  2,  the  value  overtime  of  the  average  Navy  progressor.  What  we  want 
to  know  from  this  questionnair  is  how  the  average  man  in  a specific  rating 
or  group  of  ratings  is  different  from  the  average  Navy-wide  man. 

4.  Use  the  enclosed  envelope  to  return  the  following: 

a.  DOD  8kill  Grouping  table  indicating  your  chosen  rating  group  and  ratings. 

b.  The  graph,  figure  1 or  2,  that  you  have  chosen  indicating  any  changes 
you  have  made. 

c.  An  update  of  your  Navy,  Rate/Rank  and  Mailing  Address. 

d.  Your  perferences  for  a briefing  on  the  results  of  the  experiment. 

Thanks  again  for  the  help  you  have  given  us. 
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Imjosurc  (3) 
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CONSOLIDATION  RATIONALE: 

1.  GMG  placed  in  DOD  Occupational  Area  6 with  GMM  and  GMT  due  to  the  many 
similarities  in  qualifications  and  compression  to  GM  at  paygrade  E8  and  E9- 

2.  MASTER  AT  ARJB  (MA)  established  1 August  1973* 

3.  Navy  Counselor  (NC)  established  1 August  1973* 
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AVERAGE  NAVY  PROGRESSOR 
NAME  RATE / RANK_ 

A DDRESS 

Would  you  attend  a briefing  on  the  results?  Yes No 


Figure  (2) 
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Delphi  II,  Round  3 
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M 

Paygradc  E-3 

• Value  at  LOS  0 

15 

13.2-16.4 

17 

9 

20 

• Maximum  value 

31 

30.2-32.4 

32 

32 

• LOS  reaching  maximum 

3 

2.4-  2.8 

3 

2 

• LOS  maintaining 

7 

6.5-  8.1 

5 

6 

maximum 
• Value  at  LOS  30 

10 

7.4-12.4 

12 

8 

Paygradc  E-4 

• Value  at  LOS  0 

25 

23.0-26.4 

30 

• Maximum  value 

47 

45.7-47.5 

49 

• LOS  reaching  maximum 

4 

4.1-  4.5 

5 

• LOS  maintaining 

10 

8.3-10.9 

11 

maximum 
• Value  at  LOS  30 

19 

15.8-21.6 

23 

15 

18 

Paygrnde  E-5 
• Value  at  LOS  0 

37 

34.3-38.9 

43 

29 

37 

• Maximum  value 

65 

62.9-66. 5 

70 

62 

62 

• LOS  reaching  maximum 

7 

6.8-  7.4 

7 

8 

7 

• LOS  maintaining 

13 

12.4-14.2 

14 

13 

12 

maximum 
• Value  at  LOS  30 

35 

30.4-39.0 

40 

30 

33 

Paygradc  E-6 

• Value  at  LOS  0 ' 

45 

42.8-47.2 

50 

44 

39 

• Maximum  value 

77 

76.  2-78-  6 

. 80 

76 

76 

• LOS  reaching  maximum 

10 

10.2-10.6 

11 

10 

10 

• LOS  maintaining 

18 

17.5-18.7 

18 

18 

18 

maximum 
• Value  at  LOS  30 

50 

45.3-54.7 

53 

48 

48 

Paygradc  E-7 

• Value  at  LOS  0 

51 

48.4-53.9 

54 

55 

42 

• Maximum  value 

89 

87.8-89.2 

89 

88 

88 

• LOS  reaching  maximum 

14 

14.0-14.8 

14 

15 

15 

• LOS  maintaining 

22 

21.1-22.3 

21 

22 

22 

maximum 
• Value  at  LOS  30 

66 

61.7-71.1 

65 

68 

67 

Paygradc  E-8 

• Value  at  LOS  0 

56 

52.8-58.2 

57 

60 

47 

p Maximum  value 

94 

93.3-94.5 

05 

92 

95 

• LOS  reaching  maximum 

17 

16.9-17.9 

17 

19 

17 

• LOS  maintaining 

25 

24.5-25.  7 

25 

26 

24 

maximum 
• Value  at  LOS  30 

78 

73.7-81.3 

77 

78 

78 

Paygradc  E-9 

• Value  at  LOS  0 

60 

56.6-62.4 

60 

66 

50 

• Maximum  value 

99 

98.7-99.5 

100 

98 

99 

• LOS  reaching  maximum 

20 

19.1-20.5 

19 

21 

21 

a LOS  maintaining 

27 

26.9-27.7 

27 

28 

27 

maximum 
• Value  at  LOS  30 

86 

81.7-89.5 

84 

88 

85 
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Comparison  of  Delphi  II,  Round  4,  Results 
with  Delphi  II,  Round  3,  Results  (All  Panels) 


Delphi  II,  round  3 

Delphi  II 

, round  4 

Paygrade  E-3 

• Value  at  LOS  0 

15 

15 

• Maximum  value 

31 

33 

• LOS  reaching  maximum 

3 

3 

• LOS  maintaining 

7 

7 

maximum 

• Value  at  LOS  30 

10 

10 

Paygrade  E-5 

• Value  at  LOS  0 

37 

37 

• Maximum  value 

65 

65 

• LOS  reaching  maximum 

7 

7 

• LOS  maintaining 

13 

14 

maximum 

• Value  at  LOS  30 

35 

33 

(-4%) 

Paygrade  E-7 

• Value  at  LOS  0 

51 

51 

• Maximum  value 

89 

89 

• LOS  reaching  maximum 

14 

14 

• LOS  maintaining 

22 

22 

maximum 

• Value  at  LOS  30 

66 

69 

(+3.3%) 
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